Sunday, October 25, 2009

What's missing in Btrfs

So, after being completely betrayed[1] by Ext4 not once, but twice, I decided to evaluate my FS options for /home .
  • FAT* are not an option, neither is NTFS.
  • Ext2 is primitive and HFS/HFS+ is just not Linux.
  • JFS is nice, but (atleast parted) doesn't support grow/shrink.
  • I've used XFS before, and found it to be more reliable than Ext4. However, deleting dirs with thousands of small files is too slow (a common operation when compiling)
  • ZFS would've been an option if my earlier experiences with ZFS-FUSE weren't so horrid.
  • Did not even consider NILFS. It's too new, and I don't know much about it.

It ended up being a choice between the reliable Ext3, or the new-fangled Btrfs. Why Btrfs? Because I've been using it as my Gentoo Distfiles and Portage tmpdir since v0.16, and found it to be the /most/ resilient to power failures of all my partitions.

I ended up selecting Ext3 for /home, but let's see why.

What's missing in Btrfs:

  1. Growing the filesystem to the "left" of the partition. The error message when you try this is cryptic (common in btrfs-progs). However, since for other FSes this essentially involves "move to left and grow to right", I suppose the "move" part is what's missing in btrfsctl.
  2. Pathetic ENOSPC handling. It either throws an ENOSPC at around 75% or when the metadata space fills up. Not sure which, but it's supposedly fixed for 2.6.32.
  3. Volumes once created cannot be deleted. Again, fixed in 2.6.32.
  4. Parted doesn't support editing/creating Btrfs partitions. Support for detecting it was proposed recently; but, I still don't see it in either "master" or "next". This is not a Btrfs problem, but certainly affects whether I'd use it.
  5. There were other minor irritants (with btrfs-progs, mostly), but those will go away with time

Ext3 might have bad performance (especially w.r.t fsync), but atleast it's more reliable. In conclusion, I'll use Ext3 in data=ordered mode for /home till 2.6.33 is out; and then I'll convert my Ext3 partition to Btrfs and forever be happy :}

1. betrayed == sending everything into /lost+found after a forced fsck due to an earlier fsck after a power failure

12 comments:

  1. You forgot to mention reiserfs. Not that it would necessarily be the best option but at least it's battle tested.

    ReplyDelete
  2. There are ways to make XFS delete faster:
    http://everything2.com/title/Filesystem+performance+tweaking+with+XFS+on+Linux
    I use XFS on my /home and I'm very happy with the results. The only gripe I find is it doesn't support FS shrinking, only growing.

    ReplyDelete
  3. @Betelgeuse: Right, it completely slipped my mind. However, ext* have the advantage of there being a direct migration path from them to btrfs via btrfs-convert

    @Anonymous: That link seems to just demonstrate the inverse relationship b/w speed and reliability. :)

    ReplyDelete
  4. In my case btrfs took a lot of disk space. But they said it'll be fixed.

    ReplyDelete
  5. I'm second for reiserfs3 even it's dead.
    The migration is not the main factor for me because I'm buying a new hard disk every ~3 years with double/triple capacity and prefer to reformat/recompile it (to avoid defragmentation, potential garbage and so on)

    ReplyDelete
  6. Yeah, Ext4 can really fuck you over sometimes.

    I have found small but annoying things with it. I have for example a collection of small (approx 50-100 mb big) video files laying on my server. This directory is exported over nfs3, and one day the files work, the next day while viewing it skips (with artifacts like when the file is broken) a second or two right in the beginning, and ends about one or two seconds too early.

    I could take lost+found, since then you know it was broken, but when files seems to become broken without a reason, then I start to get suspicious.

    That said, it is possible this is fixed in a more resent kernel, I have not seens it for a little while now.

    ReplyDelete
  7. You also forgot Reiser4. sys-kernel/zen-sources provides it (which is now in portage).

    I use ext4 myself though :P

    ReplyDelete
  8. For compiling I prefer a tmpfs on /var/tmp/portage. It only fails me with icedtea (and I guess openoffice, but haven't compiled that) since it's limited to 2G for me.

    I haven't had bad experiences with ext4 and I'm quite happy with it on /home so far. ext3 took ages and it's currently only a 2.3T FS, I have 930GB available in LVM so it might grow. ext4 still takes a good amount of time, but it's a lot less.

    ReplyDelete
  9. Well there's another point to this. What's missing in gentoo live-cds? Btrfs! ;)

    ReplyDelete
  10. I'm pretty sure when ext4 moved to stable... that meant that the 'on disk format' was stable. not that development of the filesystem was done. every kernel release I hear about changes to stability for ext4. I suspect it'll be several more versions until it's fully ready... and it's just about to get full testing because several distro's are moving to it.

    ReplyDelete
  11. I appreciate the work of all people who share information with others.

    ReplyDelete
  12. Hi,

    I never used ext4, but ext3 and I was disappointed, so I've made some researches to see the alternatives.

    Until btrfs will be stable enough I consider xfs to be most valuable partition type from the others available. One reason is the defragmentation option. I consider that 'Linux partition will never need to be defragmented' as a myth only.

    Anyway, I'll wait for brrfs to be included in Debian as default partition type.

    ReplyDelete