6

I'm trying to download a 40GB file and save it directly to my USB Flash Drive because I don't have enough space on my local hard drive for the file.

I am also limited to 10GB download per day on my internet connection, and my USB is formatted FAT32 and so it cannot handle files larger than 4GB.

Is there a way to download a couple 4GB chunks of this file each day and then stitch them together when they're all downloaded?

I am using Windows 10 and also have a Kali Linux live USB key.

music2myear
  • 49,799
aidn
  • 63

1 Answers1

5

If the site where you download from supports resumed downloads, you can use either use curl with the --continue-at option or wget with the --start-pos option.

While there is a --max-filesize option for curl, it just refuses to download the file.

So you can either interrupt the download when the file is large enough, or use an additional program like pv (you will probably have to install this package).

Example: Assuming "decimal" GB to be on the safe side, 4 GB = 4000000000, so use e.g.

curl --continue-at 8000000000 http://your/file/url | pv -S --size 4000000000 > your-file-name

to download the third chunk. (I hope curl handles large numbers correctly, I only checked with small numbers).

Both are standard Linux programs, I don't know if they are available for Windows as well.

dirkt
  • 17,461