I’m currently using BTRFS on most systems that I can access easily. It’s not nearly reliable enough that I want to install it on a server in another country or an embedded device that’s only accessible via 3G, but for systems where I can access the console it’s not doing too badly.
Table of Contents
Balancing and Space Allocation
# btrfs filesystem df /
Data, single: total=103.97GiB, used=85.91GiB
System, DUP: total=32.00MiB, used=20.00KiB
Metadata, DUP: total=1.78GiB, used=1.31GiB
# df -h /
| Filesystem | Size | Used | Avail | Use% | Mounted on |
| /dev/disk/by-uuid/ac696117-473c-4945-a71e-917e09c6503c | 108G | 89G | 19G | 84% | / |
Currently there are still situations where it can run out of space and deadlock on freeing space. The above shows the output of the btrfs df command and the regular df command, I have about 106G of disk space used by data and metadata in BTRFS while df shows that the entire filesystem (IE the block device) is 108G. So if I use another 2G of data or metadata then the system is at risk of deadlocking. To avoid that happening I have to run “btrfs balance start /” to start a balance which defragments the space use and frees some blocks. Currently there is a bug in BTRFS (present in all Debian/Unstable kernels) which prevents a balance operation from completing when systemd is used in a default configuration (there’s something about the way systemd accesses it’s journal files that triggers a BTRFS bug). This is really inconvenient, particularly given that there’s probably a strong correlation between people who use experimental filesystems and people who use experimental init programs.
When you get to the stage of the filesystem being deadlocked you can sometimes recover by removing snapshots and sometimes by adding a new device to the filesystem (even a USB flash drive will do). But I once had a filesystem get into a state where there wasn’t enough space to balance, add a device, or remove a snapshot – so I had to do a backup/format/restore.
Quota Groups
Last time I asked the developers (a few weeks ago) they told me that quota groups aren’t ready to use. They also said that they know about enough bugs that there’s no benefit in testing that feature. Even people who want to report bugs in BTRFS shouldn’t use quotas.
Kernel Panics with Kmail
I’ve had three systems develop filesystem corruption on files related to Kmail (the email program from KDE). I suspect that Kmail is triggering a bug in BTRFS. On all three systems the filesystem developed corruption that persisted across a reboot. One of the three systems was fixed by deleting the file for the Outbox, the others are waiting for kernel 3.14 which is supposed to fix the bug that causes kernel panics when accessing the corrupted files in question.
I don’t know whether kernel 3.14 will fix the bug that caused the corruption in the first place.
Conclusion
As I don’t use quotas BTRFS is working well for me on systems that have plenty of storage space and don’t run Kmail. There are some systems running systemd where I plan to upgrade the kernel before all the filesystem space is allocated. One of my systems is currently running SysVinit so I can balance the filesystem.
Apart from these issues BTRFS is working reasonably well for me. I haven’t yet had it’s filesystem checksums correct corrupted data from disk in any situation other than tests (I have had ZFS correct such an error, so hardware I use does benefit from this). I have restored data from BTRFS snapshots on many occasions, so that feature has been a major benefit for me. When I had a system with faulty RAM the internal checks in BTRFS alerted me to the problem and I didn’t lose any data, the filesystem became read-only and I was able to copy everything off even though it was too corrupted for writes.
I wonder how you calculated the usage of 106GB from the printed output above. Can you clarify on this, please?
103.97+1.78 is about 106G.
I was an engineer at Sun and Oracle for the last 14 years and am very familiar with zfs (love it). The biggest problem I found with customer zfs filesystems was when they were allowed to become full. The problem is due to the COW, Copy On Write, design which has many benefits, but does not tolerate a full filesystem.
My understanding is that btrfs is also a COW filesystem, so based on my experience I recommend using no more than 80% of the space on any btrfs or zfs filesystem. Going over 80% will also slow the IO down.
Peter: How does ZFS allocate space? BTRFS allocates 1GB chunks for data and 256MB chunks for metadata, this means that metadata will never be that close to the data it points to and will often be a very long way away. So I don’t think that having 20% free space is going to help much. The important thing is to have enough space to allocate new metadata chunks, I’ve had some filesystems deadlock when they run out of metadata space.