0

I have 200,000 XML files in the folder on a RHEL7 Linux server. I need to zip all 200,000 XML files. I am using the Tar command but getting the error "Argument list too long" is there any other way in this 200,000 need zip only 10,000

tar -cvf xml.tar *.xml

All xml files need to be archived separately as individual archive that has original file name in the archive name.

Original files:

1.xml
2.xml
...
n.xml

Result of archiving OP wants:

1.xml.tgz
2.xml.tgz
...
n.xml.tgz
Alex
  • 6,375
kesavanagaprasad thonta
  • 29

1 Answers1

2

You exceeded maximum command line length. Command line has finite length that you can test with getconf ARG_MAX command. When you running shell command that includes glob pattern such as * in directory that contain huge amount of files then command line is overfilled and one will receive error message "Argument list too long", so it isn't a tar problem. Keep this it in mind when you use other commands with glob patterns that applied to huge amount files.

To resolve your issue, you can use find program that will "walk"
through the directory and feed tar.

To archive all files as a single compressed tar archive you can use:

find . -name "*.xml" -print | tar -czvf xml.tgz -T -

To archive all files individually as compressed tar archives(not really sure why need to by tar'ed if it's a single file, but as you wish), use

find . -name "*.xml" -exec tar -czvf '{}'.tgz '{}' \;

To archive all files individually as gzip archives, use:

find . -name "*.xml" -exec gzip '{}' \;

Be warned, command above will remove original files (!!!)

To archive all files individually as zip archives, use:

find . -name "*.xml" -exec zip '{}'.zip '{}' \;

P.S. I added also missed(?) option to compress tar archive.

Alex
  • 6,375