Thread

Custom home server built from spare parts

91 points · 51 comments · sotilrac

Loading the complete thread in the background. This saved snapshot is available now. Refresh

[deleted] · 1789919012 · focus · HN ↗
[deleted]
bot403 · 1789923331 · focus · HN ↗
Hmmm. Only fans Linux service running?
dsk82 · 1789923892 · focus · HN ↗
Nice custom job, I use my old hardware too, it’s the perfect use case providing it’s relatively power efficient.
Not sure why they claim to have written the dashboard themselves though. It’s clearly vibe-coded, I did that too and mine is almost identical.
1. sotilrac · 1789929949 · focus · HN ↗
  Author here: totally used Claude. I meant it's a custom solution. I don't think anyone nowadays would write this type of HTML/CSS by hand; there's no point.
mindslight · 1789925148 · focus · HN ↗
[delayed]
1. close04 · 1789929597 · focus · HN ↗
  This is an underestimated part of operating IT infrastructure at home. Something a lot of homelab builders miss. They have a homelab that mixes in the same environment and same equipment the role of delivering standard services for the house and family, with the role of experimenting and toying around.
  Everyone who builds this should really have "homeprod" and "homelab". And when the experiments and tests inevitably lead to an outage, the movie streaming or photo storage or home control parts don't die with it.
  1. ismailperim · 1789935909 · focus · HN ↗
    All homelabs need a DevOps Engineer :-D
    close04 · 1789978128 · focus · HN ↗
    That doesn't save you from failure, just automates it :D.
2. sotilrac · 1789930872 · focus · HN ↗
  Absolutely, I was running the server on my main desktop before this, and it was super stressful. One of the main drivers was to move things to something more stable. NGL, I still experimenton it though...
3. sfRattan · 1789953566 · focus · HN ↗
  I very much like the name "homeprod" to contrast against "homelab."
  But I'm not as convinced as the article's author that "NAS" is outdated as a separate concept, just that as new people learn more about networking they rediscover why just having storage on in its own box can be a good setup. And when new people are brand new, having everything on one box is conceptually simpler, though not always optimal.
  I think, in the family/household context, I like the names:
  Homelab: the box/cluster you use to experiment and test new stuff.
  Homeprod: the box/cluster you use to deploy stable services for family.
  Homebase: the box/cluster you use to make ZFS pools and databases available to both homelab and homeprod (on the same network or overlay network). Basically a NAS in the older sense of the word, before homelabs started expanding it in casual use.
  Famadmin: the sysadmin for the family (i.e. "you" in the three items above). The person to whom, as you say, "providing services that support your family's larger existence, independent of the surveillance industry," is a matter of deep principle. And well put, BTW.
  1. mindslight · 1790005609 · focus · HN ↗
    [delayed]
    sfRattan · 1790017163 · focus · HN ↗
    My homebase has the around 30TB usable storage, but the only workloads I've found really need to run on the same machine (rather than the same local network) are media transcoding for Jellyfin and ML classification of footage in frigate. With those two happening, and with ZFS (appropriately) using a majority of the RAM as a cache, it makes sense to put the rest of services on another machine.
    My prod (now homeprod, thanks for the name!) has no HDDs, just SSDs. It's a proxmox cluster that runs opnsense in a VM for the whole network, Home Assistant also in a VM[1], and most other services in containers. In the age of expensive storage, it has just enough SSD space for VMs and containers. VM backups live on homebase, except for opnsense which also needs a backup on the same machine (same machine, not just same cluster) if the network goes down.
    So my conceptual distinction is mostly that the only services/containers I put on homebase are doing work that really benefits being that close to the data. Oh, and I'm in the jolly position of needing to replace the hard drives this or next year. Yaaaaay.
    My homelab is currently a cluster of 4GB Pis that I had lying around from before even the Raspberry Pi shortage. Whenever I upgrade my Framework 13 mainboard, I'll probably use the old one as a homelab instead of the Pis.
    [1]: Home Assistant's plug-ins are themselves containers, so it's a good idea to give it either a VM or its own machine.
    MayeulC · 1790022002 · focus · HN ↗
    [delayed]
    mindslight · 1790133289 · focus · HN ↗
    [delayed]
crtasm · 1789930822 · focus · HN ↗
On the wifi requirement I just noticed unraid has added wifi support
drnick1 · 1789930884 · focus · HN ↗
> So the NAS runs Kubuntu 24.04 with ZFS
My experience is similar. I no longer use dedicated NAS, firewall, or other exotic distributions. I run plain Debian, and add what I need through standard repo packages. My "NAS" is simply a Samba daemon. The same box runs countless other services including hostapd (WiFi), DNS (Unbound), an email stack (Postfix/Dovecot), multiple game servers, and Podman.
> everything else in Docker Compose
I would suggest Podman instead. It is far more secure by design (rootless) and is unable to silently alter the firewall's configuration.
1. sotilrac · 1789935593 · focus · HN ↗
  Interesting, I never tried Podman. Thanks for the suggestion!
2. Gigachad · 1789944266 · focus · HN ↗
  These days you can buy 4 bay USB-C hard drive enclosures which let you make a nas out of any mini pc. I 3D printed a 10 inch rack to put the mini pc, hard drive enclosure and switch in which keeps it all neat and looks kinda cool.
  1. magarnicle · 1789945854 · focus · HN ↗
    I do a similar thing. It really is functionally exactly the same, just a bit more piecemeal.
  2. drnick1 · 1789962745 · focus · HN ↗
    Why not an old school case with an HDD cage that can hold anywhere between a few and a dozen drives? It would be cheaper and more flexible.
    rkagerer · 1789967005 · focus · HN ↗
    Or even go nuts and do something like this: <a href="https://grabcad.com/library/10-bay-hard-disk-drive-cage-for-phanteks-enthoo-primo-case-1" rel="nofollow">https://grabcad.com/library/10-bay-hard-disk-drive-cage-for-...
  3. M95D · 1789995863 · focus · HN ↗
    USB is unreliable.
    I used btrfs raid0 on USB drives for more than a year. Every month or so, a drive would disconnect from the bus and immediately redetected as another device (sde->sdf). Btrfs does not auto-recover from this. The choices are 1) reboot and scrub, which recovers the missing drive without doing a full resync, but it needs a reboot (any hosted site is offline for 3 minutes) and scrub takes a long time reading in the background, or 2) run btrfs replace which doesn't need a reboot, but the full resync takes even longer than a scrub and increases wear if it's a ssd because it wipes everything and writes all data again.
    dsr_ · 1789996418 · focus · HN ↗
    I agree that USB is unreliable.
    I will point out that if you use ZFS instead of btrfs, you can specify drives to be detected via their serial number names rather than their attachment points, so a reconnect will not require a reboot.
    Still, don't depend on USB for anything meant to be permanently connected.
    doubled112 · 1790001612 · focus · HN ↗
    USB probably is unreliable but it's working for me. I've had zero disconnects in a couple of years of using a Terramaster USB-C DAS. I've heard of the problems enough I've even searched logs for it.
    I've wondered if my experience is different because I'm using single disks (but still ZFS) instead of any RAID. There is rarely transfer happening across all of the disks.
    M95D · 1790097874 · focus · HN ↗
    Can't say. The power source for the drives is also important. I do use a USB 2.5" HDD without an external power source for one of the SBCs I have, and that's stable, but that SBC is powered directly from the GPIO pins, not USB.
    Unfortunately, where I need it the most - the router which is also a NAS - can't use directly-powered USB HDDs. Not enough ports. I had to use a hub. I tried 4 models, two self-powered and 2 powered by the router, and none of them was stable. I also tried 3 models of USB-SATA adapters. Two self-powered, one powered by hub (connected to the hubs that had separate power - didn't work otherwise). None of them was stable.
    Then I bought a mPCIe 4-port SATA controller, Marvell chip. Didn't work. It freeze randomly and woudn't even unfreeze with a router reboot. Possibly too much power draw. Then I put a 2-port SATA, ASM1062 and a port replicator. That one works. 2 years and not one disconnect. The cables are a complete mess.
    Gigachad · 1790136249 · focus · HN ↗
    I'm using RAID on my Terramaster DAS and it's been working fine for me. I think it's less "USB is unreliable" and more "Some SATA-USB adapters are unreliable".
    The Terramaster one is pretty widely used in home setups and seems perfectly fine.
    M95D · 1790165711 · focus · HN ↗
    Be warned that Terramaster does RAID with 512B stripe size. If it fails, you can't mount the drives in Linux.
    Gigachad · 1790029046 · focus · HN ↗
    I have not had this issue myself. Drives connected for around 6 months in raid without issue. But also, you should mount your drives using partition UUID rather than whatever linux decides to name them which is not stable as you'd noticed.
3. unethical_ban · 1789948594 · focus · HN ↗
  I find Proxmox to be a great host. It's Debian with a bunch of helper scripts/GUI to run containers and VMs. I manage ZFS outside the Proxmox bubble and run a privileged SMB server container to access it.
  Though I really don't use the server anymore and it's probably worth several thousand dollars with the RAM in it. Been thinking of using a spare laptop with external HD bays to do the same job.
  1. marysol5 · 1789992281 · focus · HN ↗
    Don't know what it is with Proxmox, but I always somehow managed to break it in such a way that I can't remove the container/VM or start it...
4. Oxodao · 1789979756 · focus · HN ↗
  > is unable to silently alter the firewall's configuration
  I'd like more informations about this as I'm not too familiar with firewall setup but every time I tried to setup iptables / nft it was indeed bypassed by docker. Recently I found out in the docs that docker adds a DOCKER-USER table, isn't it enough to put your rules in it to prevent the outside world to reach containers ?
HannaTao · 1789932436 · focus · HN ↗
[dead]
yjftsjthsd-h · 1789934304 · focus · HN ↗
> So the NAS runs Kubuntu 24.04 with ZFS,
I broadly approve, but why Kubuntu? Isn't it headless?
1. sotilrac · 1789935486 · focus · HN ↗
  When I first boot up the computer I didn't know what to expect, so I defaulted to my favorite distro and it stuck
2. Lammy · 1789965012 · focus · HN ↗
  Some server-focused programs pull in UI toolkits anyway. Take qBittorrent for example: I only use its web UI, but it's still a QT program, and on my OS of choice its package will try to pull in a bunch of display-related stuff by default unless one specifically uses the “nox” package Flavor: <a href="https://cgit.freebsd.org/ports/tree/net-p2p/qbittorrent/Makefile?id=8d49a1640abdee2aefe661af292abc25cd360ab0#n39" rel="nofollow">https://cgit.freebsd.org/ports/tree/net-p2p/qbittorrent/Make...
classictraffic · 1789934949 · focus · HN ↗
That tiny touchscreen that shows metrics was an extremely nice touch. Fascinating read!
whalesalad · 1789936178 · focus · HN ↗
This is great. I also avoid stuff like TrueNAS or Unraid and opt to simply run my own ZFS and smb/nfs/etc configuration. Love the unifi-style display... I am cooking something similar but it is not a touchscreen just a blue-on-black OLED that cycles through status pages. You've inspired me to finally write a post on my own setup.
ghostly_s · 1789936682 · focus · HN ↗
I've noticed 3d printing people have this annoying habit of posting pics of their projects in just low enough res that you can't assess what the surface finish actually looks like. From what I can tell this "pebbled" finish looks uncommonly nice, I wonder if that's a byproduct of the glass fiber material or some post-processing.
1. sotilrac · 1789937836 · focus · HN ↗
  Saving that precious GitLab Pages storage . I really liked the finish of the ABS GF from Bambu Lab. It matched the original case finish pretty well out of the printer. However, the glass fill is a bit scary; you can feel the fibers poking. I wiped it with a sturdy rag to remove the loose fibers.
mrlonglong · 1789937040 · focus · HN ↗
Beware, Ubuntu are dropping ZFS. That's something that's annoyed me as that's now ruled out future upgrades
1. sotilrac · 1789937594 · focus · HN ↗
  Totally, I had to build it from source to get the latest. Also, why I chose 24.04
2. unethical_ban · 1789939382 · focus · HN ↗
  The only article I found with those keywords is that they are/were considering dropping ZFS /boot support in GRUB. If that is the case, they aren't removing ZFS support for non-boot drives.
3. fghorow · 1789947456 · focus · HN ↗
  Check out proxmox. Debian base with Debian apt repos available for standard stuff, with ZFS built into their kernels. Easy to install. And, if you insist on running TrueNAS (or whatever) just stash it in a container or KVM box.
  (Edited to add: I forgot! ZFS on root available from the installer -- encrypted even if that's your jam.)
  1. mrlonglong · 1789975747 · focus · HN ↗
    Nice! Ta.
4. rkagerer · 1789967331 · focus · HN ↗
  Do you mean it's just dropping support for booting off ZFS?
MayeulC · 1789937729 · focus · HN ↗
[delayed]
1. sotilrac · 1789938017 · focus · HN ↗
  The parts linked on the post are performing well. If you size them well, the main issue you can run into is inrush current when powering up. You may need a soft start circuit. I know I'm at the limit and may need to add a soft start if I add more fans.
  1. MayeulC · 1790022296 · focus · HN ↗
    Thanks for the feedback! A triggered power-on seauence for drives and fans makes sense, though I feel like this could also be solved by putting a couple of large capacitors on the 12V power rail (those need to charge as well, obviously, which might be an issue on startup).
2. M95D · 1789996275 · focus · HN ↗
  I have a RK3588 SBC with 5 HDDs and with a good ATX power source it draws about 24-26W from the wall socket. I tried replacing that with a 5A 12V DC power brick and the power usage increased to 28-30W.
  1. MayeulC · 1790021517 · focus · HN ↗
    [delayed]
    M95D · 1790097088 · focus · HN ↗
    3x 3.5" and 2x 2.5"
    Some power bricks are not up to the specs written on them. Get one of those USB power testers that have a barrel plug too. They're very informative in cases like these.
Lammy · 1789962023 · focus · HN ↗
Realtalk: make sure you are religious about testing your RAM if you're going to run ZFS with non-ECC RAM. It's unintuitive, but ZFS will actually modify your files on disk as a side effect of a read-only operation if blocks get read into bad RAM and checksum mismatch compared to the blocks on disk.
This happened to me in 2013–14, and it rotted a bunch of my most frequently accessed files. I was able to figure out what was going on once audible blips started appearing in all of my favorite most-listened-to music, then I learned about `flac -t` and the fact that FLAC files embed an expected MD5 digest in their `STREAMINFO` block. `man flac` sez…
“In test mode, flac acts just like in decode mode, except no output file is written. Both decode and test modes detect errors in the stream, but they also detect when the MD5 signature of the decoded audio does not match the stored MD5 signature, even when the bitstream is valid.” (emphasis mine)
```
  [Lammy@popola] flac --test &quot;Floating Door - Hypersomnia in loop - 12 templeOS.flac&quot;
  Floating Door - Hypersomnia in loop - 12 templeOS.flac: ok

  [Lammy@popola] metaflac --show-md5sum &quot;Floating Door - Hypersomnia in loop - 12 templeOS.flac&quot;
  d9ccdc78cfa45dda01b38198575b4f56

  [Lammy@popola] flac --decode --stdout --silent --force-raw-format --endian=little --sign=signed &quot;Floating Door - Hypersomnia in loop - 12 templeOS.flac&quot; | md5
  d9ccdc78cfa45dda01b38198575b4f56
```
1. sotilrac · 1790000383 · focus · HN ↗
  Author here: I've been taking a lot of interest in data corruption and preservation lately. Definitely will look into this. Thanks for the pointer!
justsomehnguy · 1789967089 · focus · HN ↗
> six drives spinning up at once pull about 125 W for a brief moment
<a href="https://en.wikipedia.org/wiki/Spin-up#SATA_and_PATA" rel="nofollow">https://en.wikipedia.org/wiki/Spin-up#SATA_and_PATA
Of course it's a lottery if it would work on a kitbashed homelab case.
kolli_kumar · 1789985920 · focus · HN ↗
It’s a HDD, so it’s ok if not CoW will kill your consumer SSD/NVMe sooner.
kolli_kumar · 1789986030 · focus · HN ↗
Also you can very well add Proxmox if you want to on top of Debian.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.