How to create multiple ZIP files in a folder based on 4GB size restriction?

Question

I have a directory which contains files and I need to create individual ZIP files of these files based on a 4GB size restriction without creating a multipart ZIP archive. Could someone help here how to achieve this?

I knew about passing the -s parameter to the ZIP command (zip -s 4096m zipfile.zip *.xml) but htis will create a multipart ZIP which I don't need it.

And I need to split the ZIP files based on the 4GB size with incremental ZIP file names like if I do zip -s 4096 archive.zip *.xml. It should create ZIP files like archive.zip, archive1.zip with size of 4GB split etc. in the same directory but not the multipart ZP files like archive.zip, archive.z01 etc..

You have to give more details: what kind of resulted file to you want? What are the target and the goal. — F. Hauri - Give Up GitHub, Jan 19 '22 at 07:18
With a temporary file, you could add file by file, by using `tar --append -f tarballtest.tar "$file"`, while `[[ $(zstd — F. Hauri - Give Up GitHub, Jan 19 '22 at 07:34
I need to do split the zip files based on the 4GB size with incremental zip file name like if I do zip -s 4096 archive.zip *.xml should create zip as archive.zip, archive1.zip with size of 4gb split etc in the same directory not the multipart zip like archive.zip, archive.z01 etc. — NVNK, Jan 19 '22 at 07:49
Ok it's approx what I've guessed. So yes: You'd better using `tar | zstd`, for quickness and efficience. See my previous comment about kind of `while` loop... — F. Hauri - Give Up GitHub, Jan 19 '22 at 07:56
Insted of adding details in comments, edit your question! Add clearly that the goal is to be able to 1. index your archives files and 2. be able to access directly one archive file without the need of reconscruct whole archive. — F. Hauri - Give Up GitHub, Jan 19 '22 at 07:59
So you could do something same, by successive tries, but as you have to re-build each zip until size reach limit, this will take a lot of resources and time!!! The more: using `zstd` instead of `gzip` or `zip` will reduce resource consumtion by approcx `7/8`!! — F. Hauri - Give Up GitHub, Jan 19 '22 at 14:35

U880D · Answer 1 · 2023-03-08T02:47:26.773

I understand your question as follow

There is a directory with a variable amount of XML files
Zipping all files via zip zipfile.zip *.xml would result into a ZIP file bigger than 4GB
You don't want or can't use multipart ZIP zip -s 4096m zipfile.zip *.xml because of How to unzip a multipart (spanned) ZIP
You don't want or can't use the split command to split one huge file because you would need to join files after splitting also
You don't want or can't use tar either, even if it is possible to open a .tar.gz file with a single command or to unpack or extract files

but you want to create ZIP files with size limit.

The command zipslit

reads a zipfile and splits it into smaller zipfiles.

whereby the parameter

  -n size
         Make zip files no larger than "size" (default = 36000).

So, to split the ZIP files do a test with

zip zipfile.zip *.xml
zipsplit -t zipfile.zip

I could see zipsplit has limitation of splitting the 2GB[https://www.computerhope.com/unix/zipsplit.htm] file only in my case it should be 4GB file but I resolved my issue using zipsize counter in loop when its reached it will increment the zip name with seq. — NVNK, Jan 27 '22 at 07:18

How to create multiple ZIP files in a folder based on 4GB size restriction?

1 Answers1