Thread

Btrfs/ZFS/bcachefs under workloads classic benchmarks skip

179 points · 190 comments · farlight

lproven · 1789904202 · focus · HN ↗
Interesting although I'd have liked more summaries: there's an awful lot there.
But the reasons I choose filesystems are more about reliability, failure modes, surrounding tooling, and so on.
Btrfs fails in several critical areas:
1. No way to accurately find free space
2. catastrophic failure on write if a volume fills up, the probability of which is greater because of #1
3. repair tools usually do not recover a corrupted volume and in my testing are most likely to render as damaged volume completely unreadable, which makes #2 worse
Put these things together and I can never trust Btrfs again. In the 9 years since I encountered these, I see no effort to fix them, just fooling around witg unimportant side details like performance tweaks.
Fix the critical issues first then make it faster.
1. raegis · 1789916803 · focus · HN ↗
  Does the report say any of this? I only see "FAIL" on a few tests with ext4 and one with xfs.
  1. koverstreet · 1789919306 · focus · HN ↗
    It's hard to show with any accuracy how likely a filesystem is to not break when the SHTF or something weird happens, or if they've handled all the weird corner cases, with any kind of automated test.
    For that you have to dig into the methodology, look at the code, look at user reports, etc.
    But you can get a pretty good approximation just from the philosophies and attitudes of the engineers and what they're talking about.
    The talk I just gave at the Rust for Linux conference was all about that - how do we make the system debugable, the community aspect of how we respond to bug reports and talk to users, the prep work for the Rust conversion and formal verification and how we're approaching all that.
    Reliability doesn't come out of nowhere, "all bugs are shallow with enough eyeballs" really doesn't apply to filesystems. You just have to plan for it, come up with a methodology, and do the work.
  2. lproven · 1789930875 · focus · HN ↗
    It's not just me and it's had serious problems for years:
    <a href="https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/" rel="nofollow">https://arstechnica.com/gadgets/2021/09/examining-btrfs-linu...
    <- 5Y ago.
    It's not materially better now. The devs are in denial about the problems because lots of big users are saying "works fine on my machine."
    Sure, if you have lots of backups, if you have huge volumes on huge disks and they never fill up...
    But it's the default in Fedora, Spiral Linux, Garuda Linux, siduction and others. Personal distros for people's own PCs and those are not well-supported enterprise kit.
    chasil · 1790012317 · focus · HN ↗
    SUSE also famously uses btrfs on the root file system.
    Don't ever let it fill up!
    lproven · 1790077776 · focus · HN ↗
    Exactly.
    SUSE takes snapshots before packages are installed. The package manager cannot tell if the disk will fill up as a result because on Btrfs the `df` command lies.
    If its Btrfs root fills up and the OS writes to it, it 100% will self-destruct, and the `btrfs-repair` tool (`fsck` replacement) cannot fix drives and usually makes the corruption worse and renders the drive unreadable.
    This is why in the earlier thread about swapfiles...
    <a href="https://news.ycombinator.com/item?id=49618087">https://news.ycombinator.com/item?id=49618087
    ... I advised 2 commenters not to recommend keeping the OS and data in a single big volume. I got downvoted for it. I was rude. Well, I was, but I stand by my comments even though I'm sorry for my tone when I made them.
    chasil · 1790117210 · focus · HN ↗
    You were absolutely correct to be rude.
    A pragmatic fix would trigger read-only failure mode at 90% usage of the mount as a whole, with all subvolumes.
    lproven · 1790259470 · focus · HN ↗
    Thanks!
    And you know what, that's a great idea.
    chasil · 1790309890 · focus · HN ↗
    The more practical advice is to resize hard partitions down by 5-10%.
    I have a loopback mount on an fallocate file with a btrfs filesystem inside of it at work. I'm using send/receive to make sure there is hope of recovery.
    I wrote this several years ago.
    <a href="https://www.linuxjournal.com/content/btrfs-centos-living-loopback" rel="nofollow">https://www.linuxjournal.com/content/btrfs-centos-living-loo...
    lproven · 1790338469 · focus · HN ↗
    That is excellent stuff -- thank you!
2. sandreas · 1789917100 · focus · HN ↗
  This. Btrfs blew up without ANY reason at all in my case. Rebooted and the system won't even recognize any filesystem. All btrfs tools fails to recover a single file.
  Here is my journey: <a href="https://forum.cgsecurity.org/phpBB3/viewtopic.php?p=39143" rel="nofollow">https://forum.cgsecurity.org/phpBB3/viewtopic.php?p=39143
  I switched to ZFS and never Bad a Problem again.
  1. burnt-resistor · 1789978376 · focus · HN ↗
    I used ZFS until it self-corrupted and refused to mount rw ever again. Community support was totally unhelpful and there was no resolution except buy another array and use something else. There's way too much ZFS cult fanboy glazing out there it doesn't deserve.
    lproven · 1790002839 · focus · HN ↗
    Now that's something I've not heard of before. On what OS? What happened, as far as you know or can tell?
    It would mount read-only, then? So you could copy your stuff off onto other media? Because if so, it's better than my experiences with Btrfs.
  2. FrinkleFrankle · 1790029895 · focus · HN ↗
    I've had a ZFS array running since ~2009. It's gone through upgrades and disk replacements, etc. Have not suffered a data loss in that time. ZFS I'd the goat.
3. KrOctave · 1789918793 · focus · HN ↗
  This is why I have fully switched to Bcachefs, which is just way more stable and performant than btrfs

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.