9

I'm looking for a file system suited for my storage task, which is to store between 300 and 400 gigs of media data, i.e. video, sound and pictures.

The data will be stored on a non-SSD drive which will be used only for that data, i.e. boot time on the file system is a non-issue. The surrounding system will be running Debian (or, in the future, maybe some other Linux).

I don't think I can feed the file system data faster than the network can deliver it, so write speeds probably don't need to exceed 1-10 MB/s for the foreseeable future. Also, I suspect that writes will be fairly infrequent, so I might like to mount the partition read-only and then remount when I want to write.

Primarily I want to read at 1 x speed, i.e. read one second's worth of video in one second, though I might like to seek also; I don't know enough about video formats to understand what kind of throughput this requires. I suspect it requires that an index of key frames are in RAM and that it is cheap to jump to any disk block which contains a key frame (i.e. it's in RAM too); on top of this, the disk plus file system must be able to stream video data at 1 x speed starting with only the key frame block in RAM.

The videos are private so I would like to encrypt the data. Also, I would hate to lose any of the videos; given that my throughput requirements are low, I'm willing to sacrifice performance for stability. Naturally, since video formats already compress the data, I don't care about support for file system level compression. The way my home is configured, there's a moderate to high risk that I hit the power switch by accident and I have no UPS.

My questions:

  • Have I understood correctly what I should look for in a file system primarily meant for storing and accessing video data? In particular, if you have used a storage system for media data, what surprised you the most? What did you learn on the way that you couldn't have foreseen? What can I learn from your experience?

  • Assuming that encryption support, rock solid stability and moderate performance are the criteria by which I really want to evaluate file systems, do you know which file system would be best in my situation? Do you know of any relevant data about this issue?

Jonas Kölker
  • 323
  • 3
  • 6

1 Answers1

4

Your requirements are not particularily demanding or difficult to meet. I would suggest simply using your favorite general purpose filesystem (be it ext4 or xfs or anything else, really) with dm-crypt underneath to support the encryption.

Choice of filesystem

I'm sure you will find lots of studies out there that say filesystem X is better than filesystem Y for use case Z, and others that say the opposite. You are interested in reliability over performance so you should use one that's very widely used and has a lot of history behind it, which means that most of the bugs have (hopefully) been worked out. xfs and ext4 are both good candidates. ext4 is younger but it's an incremental upgrade from ext3 so maybe that doesn't count for much. xfs has better features than ext3 (like freezing for LVM snapshots) but I think ext4 probably evens the playing field. In the end? Probably pick ext4 as it's what most Linux distribution installers default to nowadays.

dm-crypt

dm-crypt operates on the block device level so you can use any filesystem you want on top of it. Use cryptsetup's luks mode. It's easier to use than cryptsetup's other mode.

Celada
  • 2,390