8

I recently bought a 8TB HDD to replace my 1TB drive for storing my backups. However after moving the files to the new drive I noticed they take up a substantially larger amount of space on the disk, even though the true file size is the same. Both drives are formatted to be exFAT so I am not sure why the reported size is so different. Why is substantially more space being used on the new drive?

File size comparison (Left is old, right is new)

phuclv
  • 30,396
  • 15
  • 136
  • 260
Oh Dear
  • 105

2 Answers2

12

The default cluster size for exFAT is extremely large:

Volume size         Cluster size
7 MB to <256 MB     4 KB
256 MB to <32 GB    32 KB
32 GB to <512 GB    128 KB
512 GB to <1 TB     256 KB
1 TB to <2 TB       512 KB
2 TB to <4 TB       1 MB
4 TB to <8 TB       2 MB
8 TB to <16 TB      4 MB
16 TB to <32 TB     8 MB
32 TB to <64 TB     16 MB
64 TB to <512 TB    32 MB

Your files are 753,674,490,827 ÷ 726,245 ≈ 0.9897 MB on average, so on the 1000GB drive (smaller than 1 TB) with default 256-KB cluster, about half of the files take 4 clusters (and waste almost nothing) and roughly the remaining half need 5 clusters (and waste 1 cluster)

The average wastage is 0.5 clusters = 128 KB per file or 12.5%. The total expected size on disk is 753,674,490,827 × 112.5% ≈ 789.7 GB which is extremely close to what's in your screenshot. However on the 8000GB disk, the clusters are 2 MB, so you're wasting ~1 MB per file, which is why the new size on disk is almost twice the total size and the previous size on disk

In short, exFAT is bad for storing small files on large partitions by default. Either use NTFS, or reformat exFAT with a smaller cluster size (4 KB just like almost everyone else including NTFS)

See

It's also the same reason as this question Why is there such a big difference between "Size" and "Size on disk"?

On NTFS there are many other reasons. If you're interested read Why does "Size on disk" vary when "Size" does not with the same set of files?

phuclv
  • 30,396
  • 15
  • 136
  • 260
3

Phuclv is right. If you are using ExFAT on a 8TB disk your cluster size will be 8MiB which means that any file will take at least 8MB of disk space, even if it's only a small txt file.

For a drive that big I would recommend ReFS (if you are using Windows 10 or 2012/2016 server) with a 4KB cluster size:

ReFS Cluster Size

or if you are looking for an even smaller cluster size go for NTFS:

NTFS Cluster Size

phuclv
  • 30,396
  • 15
  • 136
  • 260
NBorba
  • 31
  • 1