133

In the informal (i.e. journalistic) technology press, and in online technology blogs and discussion forums, one commonly encounters anecdotal advice to leave some amount of space free on hard disk drives or solid state drives. Various reasons for this are given, or sometimes no reason at all. As such, these claims, while perhaps reasonable in practice, have a mythical air about them. For instance:

  • “Once your disk(s) are 80% full, you should consider them full, and you should immediately be either deleting things or upgrading. If they hit 90% full, you should consider your own personal pants to be on actual fire, and react with an appropriate amount of immediacy to remedy that.” (Source.)

  • “To keep the garbage collection going at peak efficiency, traditional advice is aim to keep 20 to 30 percent of your drive empty.” (Source.)

  • “I've been told I should leave about 20% free on a HD for better performance, that a HD really slows down when it's close to full.” (Source.)

  • “You should leave room for the swap files and temporary files. I currently leave 33% percent free and and vow to not get below 10GB free HDD space.” (Source.)

  • “I would say typically 15%, however with how large hard drives are now adays, as long as you have enough for your temp files and swap file, technically you are safe.” (Source.)

  • “I would recommend 10% plus on Windows because defrag won't run if there is not about that much free on the drive when you run it.” (Source.)

  • “You generally want to leave about 10% free to avoid fragmentation.” (Source.)

  • “If your drive is consistently more than 75 or 80 percent full, upgrading to a larger SSD is worth considering.” (Source.)

Has there been any research, preferably published in a peer-reviewed journal, into either the percentage or absolute amount of free space required by specific combinations of operating systems, filesystem, and storage technology (e.g. magnetic platter vs. solid state)? (Ideally, such research would also explain the reason to not exceed the specific amount of used space, e.g. in order to prevent the system running out of swap space, or to avoid performance loss.)

If you know of any such research, I would be grateful if you could answer with a link to it plus a short summary of the findings. Thanks!

Giacomo1968
  • 58,727

9 Answers9

51

Though I can't talk about "research" being published by "peer reviewed journals" - and I wouldn't want to have to rely on those for day-to-day work - I can though talk about the realities of hundreds of production servers under a variety of OSes over many years:

There are three reasons why a full disk reduces performance:

  • Free space starvation: Think of temp files, Updates, etc.
  • File system degradation: Most file systems suffer in their ability to optimally lay out files if not enough room is present
  • Hardware level degradation: SSDs and SMR disks without enough free space will show decreased throughput and - even worse - increased latency (sometimes by many orders of magnitude)

The first point is trivial, especially since no sane production system would ever use swap space in dynamically expanding and shrinking files.

The second point differs highly between file systems and workload. For a Windows system with mixed workload, a 70% threshold turns out to be quite usable. For a Linux ext4 file system with few but big files (e.g. video broadcast systems), this might go up to 90+%.

The third point is hardware and firmware dependent, but especially SSDs with a Sandforce controller can fall back in free-block erasure on high-write workloads, leading to write latencies going up by thousands of percent. We usually leave 25% free on the partition level, then observe a fill rate of below 80%.

Recommendations

I realize that I mentioned how to make sure a max fill rate is enforced. Some random thoughts, none of them "peer reviewed" (paid, faked or real) but all of them from production systems.

  • Use filesystem boundaries: /var doesn't belong into the root file system.
  • Monitoring, monitoring, monitoring. Use a ready-made solution if it fits you, else parse the output of df -h and let alarm bells go of in case. This can save you from 30 kernels on a root fs with automatic-upgrades installed and running without the autoremove option.
  • Weigh the potential disruption of a fs overflow against the cost of making it bigger in the first place: If you are not on an embedded device, you might just double those 4G for root.
Community
  • 1
Eugen Rieck
  • 20,637
30

Has there been any research...into either the percentage or absolute amount of free space required by specific combinations of operating systems, filesystem, and storage technology...?

In 20 years of system administration, I've never encountered research detailing the free space requirements of various configurations. I suspect this is because computers are so variously configured it would be difficult to do because of the sheer number of possible system configurations.

To determine how much free space a system requires, one must account for two variables:

  1. The minimum space required to prevent unwanted behavior, which itself may have a fluid definition.

    Note that it's unhelpful to define required free space by this definition alone, as that's the equivalent of saying it's safe to drive 80 mph toward a brick wall until the very point at which you collide with it.

  2. The rate at which storage is consumed, which dictates an additional variable amount of space up be reserved, lest the system degrade before the admin has time to react.

The specific combination of OS, filesystems, underlying storage architecture, along with application behavior, virtual memory configuration, etc. creates quite the challenge to one wishing to provide definitive free space requirements.

That's why there are so many "nuggets" of advice out there. You'll notice that many of them make a recommendation around a specific configuration. For example, "If you have an SSD that's subject to performance issues when nearing capacity, stay above 20% free space."

Because there is no simple answer to this question, the correct approach to identify your system's minimum free space requirement is to consider the various generic recommendations in light of your system's specific configuration, then set a threshold, monitor it, and be willing to adjust it as necessary.

Or you could just keep at least 20% free space. Unless of course you have a 42 TB RAID 6 volume backed by a combination of SSDs and traditional hard disks and a pre-allocated swap file... (that's a joke for the serious folks.)

Community
  • 1
I say Reinstate Monica
  • 26,651
26

Has there been any research, preferably published in a peer-reviewed journal […]?

One has to go back a lot further than 20 years, of system administration or otherwise, for this. This was a hot topic, at least in the world of personal computer and workstation operating systems, over 30 years ago; the time when the BSD people were developing the Berkeley Fast File System and Microsoft and IBM were developing the High Performance File System.

The literature on both by its creators discusses the ways that these filesystems were organized so that the block allocation policy yielded better performance by trying to make consecutive file blocks contiguous. You can find discussions of this, and of the fact that the amount and location of free space left to allocate blocks affects block placement and thus performance, in the contemporary articles on the subject.

It should be fairly obvious, for example, from the description of the block allocation algorithm of the Berkeley FFS that, if there is no free space in the current and secondary cylinder group and the algorithm thus reaches the fourth level fallback ("apply an exhaustive search to all cylinder groups"), performance of allocating disc blocks will suffer as also will fragmentation of the file (and hence read performance).

It is these and similar analyses (these being far from the only filesystem designs that aimed to improve on the layout policies of the filesystem designs of the time) that the received wisdom of the past 30 years has built upon.

For example: The dictum in the original paper that FFS volumes be kept less than 90% full, lest performance suffer, which was based upon experiments made by the creators, can be found uncritically repeated even in books on Unix filesystems published this century (e.g., Pate2003 p. 216). Few people question this, although Amir H. Majidimehr actually did the century before, saying that xe has in practice not observed a noticeable effect; not least because of the customary Unix mechanism that reserves that final 10% for superuser use, meaning that a 90% full disc is effectively 100% full for non-superusers anyway (Majidimehr1996 p. 68). So did Bill Calkins, who suggests that in practice one can fill up to 99%, with 21st century disc sizes, before observing the performance effects of low free space because even 1% of modern size discs is enough to have lots of unfragmented free space still to play with (Calkins2002 p. 450).

This latter is an example of how received wisdom can become wrong. There are other examples of this. Just as the SCSI and ATA worlds of logical block addressing and zoned bit recording rather threw out of the window all of the careful calculations of rotational latency in the BSD filesystem design, so the physical mechanics of SSDs rather throw out of the window the free space received wisdom that applies to Winchester discs.

With SSDs, the amount of free space on the device as a whole, i.e., across all volumes on the disc and in between them, has an effect both upon performance and upon lifetime. And the very basis for the idea that a file needs to be stored in blocks with contiguous logical block addresses is undercut by the fact that SSDs do not have platters to rotate and heads to seek. The rules change again.

With SSDs, the recommended minimum amount of free space is actually more than the traditional 10% that comes from experiments with Winchester discs and Berkeley FFS 33 years ago. Anand Lal Shimpi gives 25%, for example. This difference is compounded by the fact that this has to be free space across the entire device, whereas the 10% figure is within each single FFS volume, and thus is affected by whether one's partitioning program knows to TRIM all of the space that is not allocated to a valid disc volume by the partition table.

It is also compounded by complexities such as TRIM-aware filesystem drivers that can TRIM free space within disc volumes, and the fact that SSD manufacturers themselves also already allocate varying degrees of reserved space that is not even visible outwith the device (i.e., to the host) for various uses such as garbage collection and wear levelling.

Bibliography

AJM
  • 500
JdeBP
  • 27,556
  • 1
  • 77
  • 106
13

Of course, a drive itself (HDD or SSD alike) couldn't care less about how many percents of it are in use, apart from SSDs being able to erase their free space for you beforehand. Read performance will be exactly the same, and write performance may be somewhat worse on SSD. Anyway, write performance is not that important on a nearly full drive, since there's no space to write anything.

Your OS, filesystem and applications on the other hand will expect you to have free space available at all times. 20 years ago it was typical for an application to check how much space you had on the drive before trying to save your files there. Today, application create temporary files without asking your permission, and typically crash or behave erratically when they fail to do so.

Filesystems have a similar expectation. For example, NTFS reserves a big chunk of your disk for MFT, but still shows you this space as free. When you fill your NTFS disk above 80% of its capacity, you get MFT fragmentation which has a very real impact on performance.

Furthermore, having free space does indeed help against fragmentation of regular files. Filesystems tend to avoid file fragmenation by finding the right place for each file depending on its size. On a near-fill disk they'll have fewer options, so they'll have to make poorer choices.

On Windows, you're also expected to have enough disk space for the swap file, which can grow when necessary. If it can't, you should expect your apps getting forcibly closed. Having very little swap space can indeed worsen the performance.

Even if your swap has fixed size, running completely out of system disk space can crash your system and / or make it unbootable (Windows and Linux alike), because the OS will expect to be able to write to disk during booting. So yes, hitting 90% of disk usage should make you consider your paints being on fire. Not once have I seen computers which failed to boot properly until recent downloads were removed to give the OS a little disk space.

Dmitry Grigoryev
  • 9,861
8

For SSDs there should be some space left because rewrite rate then increases and negatively affects write performance of the disk. The 80% full is safe value probably for all SSD disks, some latest models may work fine even with 90-95% occupied capacity.

https://www.howtogeek.com/165542/why-solid-state-drives-slow-down-as-you-fill-them-up/

Jaroslav Kucera
  • 1,562
8

The "rules" vary depending on your requirements. And there are special cases, like, for example ZFS: "At 90% capacity, ZFS switches from performance- to space-based optimization, which has massive performance implications.". Yes, this is a design aspect of ZFS...not something derived via observation or anecdotal evidence. Obviously, this is less of an issue if your ZFS storage pool consists solely of SSDs. However, even with spinning disks, you can happily reach 99% or 100% when you deal with static storage and you don't need top-notch performance - for example, your personal all-time favorite movie collection, which never changes and where safety is priority 1.

Next, btrfs - an extreme case: when free space gets too low (a few MByte), you can hit the point of no return. No, deleting files is not an option, as you can't. There is simply not enough space to delete files. btrfs is a COW (copy-on-write) file system, and you can reach a point where you cannot modify metadata any more. At this point, you can still add additional storage to your file system (a USB thumb drive might work), then delete files from the expanded file system, then shrink the file system and remove the additional storage again). Again, this is some aspect caused by the design of the file system.

People who can give you "real (serious) data" are probably the ones which deal with "real (serious) storage". Twisty's (excellent) answer mentiones hybrid arrays (consisting of massive amounts of cheap slow spinning, lots of fast spinning disks, many SSDs...) which are operated in an enterprise environment where the major limiting factor is the speed at which the admin is able to order upgrades. Going from 16T to 35T can take 6 months...so you end up with seriously backed reports which suggest to set your alarm to 50%.

Klaws
  • 324
4

One thing to consider with mechanical drives is that the throughput of the outer edge is higher than the inside. This is because there are more sectors per revolution for the larger circumference of the outside.

As the drive reaches capacity, perfomance will decrease because only slower inner sectors will be available.

For a more in depth analysis see https://superuser.com/a/643634

Wes Toleman
  • 231
4

There are many, many factors that contribute to the result in very setup-specific amounts. So, there's no hard-and-fast number, this can only be measured as a function of those parameters. (That's probably why other users report no specific research on this specific subject made -- there are too many variables to compile anything conclusive.)

  • Hardware

    • HDD has all its sectors allocated at all times. So it absolutely doesn't care how many of them contain current user data. (For the controller, all sectors contain some data at all times, it just reads and overwrites them as told to.)
    • SSD's controller, on the other hand, (de)allocates its sectors dynamically, akin to a file system. Which makes this job harder at higher usages. How much harder it is and how much this affects observable performance depends on:
      • The controller's performance and algorithms' quality
        • In particular, whether it supports the TRIM command (some cheap SSDs don't)
      • Write load
      • To a lesser extent, overall load (to give the controller time for garbage collection)
      • Space overprovision (this is when an SSD has physically significantly more space than it advertises to the OS. Some manufacturers of high-end SSDs even let the customer choose the overprovision percentage pre-order or change it dynamically)

  • File system

    • Different file systems are designed for different loads and host processing requirements. This can be tweaked to some extent by format parameters.
    • FS write performance is a function of free space and fragmentation, read performance is a function of fragmentation only. It degrades gradually from the very beginning, so the question is where your tolerable threshold lies.

  • Load type

    • Write-heavy load emphasizes quickly finding and accessing new free blocks
    • Read-heavy load emphasizes consolidating related data so it can be read with less overhead
ivan_pozdeev
  • 1,973
3

It depends on the intended usage of the drive, but in general 20% to 15% free space is a good answer for spinning disks, and 10% or more is good for SSDs.

If this is the main drive on the computer and files may be moved, then 20% free space should prevent significant slowdowns. This will allow enough open space throughout the drive for data to be moved and copied as needed. A spinning drive will work best when the free locations are nearer to the original data, while in an SSD the physical location doesn't affect daily performance. So, the spinning drive should have more free space for purely performance reasons. On the SSD, the reduced free space will reduce drive longevity, but it won't reduce performance. SSD's try to store temporary data and random download files in least-used locations so they can balance the cell usage across the entire drive; otherwise a portion of the drive will age much faster than the rest of it. Some newer SSDs may intelligently re-arrange files so the empty space moves, but older drives may not do this.

If this is a media or long-term storage drive, then 5% to 10% free should be sufficient, and 10% would be preferable if it is a spinning disk. You don't need as much free space because this drive will rarely require data to be moved, so performance isn't nearly as much of a factor. The free space is useful mainly to allow bad sectors to be discarded and replaced, and to allow files to be more contiguous.

I wouldn't push any drive past 95% capacity for more than a day unless there is a very good, clear reason.

user3685427
  • 147