6

I've read in this answer https://superuser.com/a/1358858/1506333 that NTFS does double writes, uncompressed data first, then compressed, is it true?

Motive NÂș1: It will kill SSD much faster, since it makes two writes; NTFS compression always writes uncompressed data prior to start compression on RAM and then re-write compressed data only if it is a gain of at least 4KiB.

also these

1 Answers1

12

I was also interested in this, and couldn't find any source other than people making this claim. So time for some original research.

TL;DR

At least for Windows 10,

  • the claim "NTFS compression writes file data twice" is BUSTED
  • the claim "NTFS compression always leads to highly fragmented files" is BUSTED

Details of what I did:

I ran a fully updated Windows 10 in a KVM virtual machine and attached a new, empty, emulated 15GB SSD with disabled caching and instrumented KVM to measure the actual writes issued by windows to the hardware.

(SI units unless noted)

  • Result 0a: the raw sparse disk image started out using 2MB
  • Result 0b: partitioning/formatting the drive wrote 49MB
  • Result 0c: the disk image grew to 53M

Experiment 1: copy a 1GiB file consisting of binary zeroes from a samba share to the newly created disk (by dragging in explorer).

  • Result 1a: this physically wrote 3650kB to the disk
  • Result 1b: disk image did not grow
  • Result 1c: file propoerties show that the file takes up 0 bytes
  • Result 1d: contig shows file as fully defragmented

Experiment 2: delete the file, copy a 1GB file consisting of highly compressible data (0x790a repeated), using windows explorer.

  • Result 2a: this physically wrote 72MB to the disk
  • Result 2b: disk image did grow to 116MB
  • Result 2c: file properties show that the file takes up 63MB
  • Result 2d: contig shows file as using 2 fragments

Experiment 3: disable compression, copy the same file again to the disk.

  • Result 3a: this physically wrote 1003MB to the disk
  • Result 3b: disk image did grow to 1.5GB
  • Result 3c: file properties show that the file takes up 1000MB
  • Result 3d: contig shows file as using 1 fragment

Experiment 4: compress this file using explorer properties.

  • Result 4a: this physically wrote 75MB to the disk
  • Result 4b: file properties show that the file takes up 63MB
  • Result 4c: contig shows file as using 4 fragments
  • Result 4d: contig shows drive as having 4 free space fragments

Experiment 5: created a 5GB disk and partition, copy a 10GB highly compressible file to it, using "type" (explorer/copy immediately refuse as the destination drive is not large enough).

  • Result 5a: this aborted after writing 4600MB
  • Result 5b: this physically wrote 397MB to the disk
  • Result 5c: disk image did grow to 548MB
  • Result 5d: file properties show that the file takes up 280MB
  • Result 5e: contig shows file as using 57 fragments
  • Result 5f: contig shows drive as having 24 free space fragments

Overall observation 1: when writing the files, windows started writing relatively fast (5MB/s), but after the copy was "finished" it kept writing for more than 40s at ever decreasing speeds (500kB/s down to 54kB/s in the last few seconds). This is a bit bizarre, and might lead credence to the theory that something happens in the background after writing the file.

Overall observation 2: even simple commands such as "type" are shockingly slow.

Overall observation 3: windows apparently recognizes either chunks with zeroes when compression, or auto-detects runs of zeroes as sparse files (The source files were not sparse). This is surprising.

Cororally: in the (recent) past, I have found that files acquired millions of fragments with NTFS compression (mostly unity "WindowsNoEditor" game pack files), so NTFS does sometimes do this, but this seems to be limited to certain files and/or certain conditions, and is generally a rare occasion (I found 15 out of 1949979 files on my 3TB compressed disk to show this pattern).

Remember Monica
  • 745
  • 7
  • 11