Comment by Havoc

Comment by Havoc 9 hours ago

10 replies

My most recent learning - DDR4 ECC UDIMMs are comically expensive. To the point where I considered just replacing the entire platform with something RDIMM rather than swapping to ECC sticks.

>No space left on device.

>In other words, you can lock yourself out of PBS. That’s… a design.

Run PBS in LXC with the base on a zfs dataset with dedup & compression turned off. If it bombs you can increase disk size in proxmox & reboot it. Unlike VMs you don't need to do anything inside the container to resize FS so this generally works as fix.

>PiHole

AGH is worth considering because it has built in DoH

>Raspberry Pi 5, ARM64 Proxmox

Interesting. I'm leaning more towards k8s for integrating pis meaningfully

everforward 5 hours ago

You seem knowledgeable so you may already know, but it's worth looking at the x86 mini PCs. Performance per watt has gotten pretty close on the newer low power CPUs (e.g. N150, unsure what AMD's line for that is), and performance per $ spent on hardware is way higher. I'm seeing 8GB Pi 5s with a power supply and no SD card for $100; you can get an N150 mini PC with 16GB of RAM and 500GB SSD pre-installed for like $160. Double the RAM, double the CPU performance, and comes with an SSD.

Imo, Raspberry Pis haven't been cost competitive general compute devices for a while now unless you want GPIO pins.

  • Havoc 5 hours ago

    Yeah have a collection of minipc - they are indeed great. This build was more NAS focused. 9x SATA SSD and 6x NVME...minipcs just don't have the connectivity for that sort of thing

    >Imo, Raspberry Pis haven't been cost competitive general compute devices for a while now unless you want GPIO pins.

    I have a bunch of rasp 4Bs that I'll use for a k8s HA control plane but yeah outside of that they're not idea. Especially with the fragility of SD card instead of nvme (unless you buy the silly HAT thing).

    • heresie-dabord 21 minutes ago

      > Raspberry Pi 5s can actually support NVME drives

      And Raspberry Pi 4s can actually boot from NVME via a USB enclosure.

  • dangus 4 hours ago

    The first thing I thought when I read this article was how raspberry pi’s just make this kind of thing more difficult and annoying compared to a regular normal PC, new (e.g. cheap mini PC) or used (e.g. used business workstation or just a plain desktop PC).

    And if you want GPIO pins I’d imagine that a lot of those applications you’d be better served with an ESP32 and that a raspberry pi is essentially overkill for many of those use cases.

    The Venn diagram for where the pi makes sense seems smaller than ever these days.

    • perdomon 2 hours ago

      You're right that the Venn diagram is smaller than it was 5 years ago, but there are still some folks whose primary concern is electricity usage. Even the pi 5 shines there (as long as you don't need too much compute).

Aurornis 8 hours ago

> My most recent learning - DDR4 ECC UDIMMs are comically expensive. To the point where I considered just replacing the entire platform with something RDIMM rather than swapping to ECC sticks.

DDR4 anything is becoming very expensive right now because manufacturers have been switching over to DDR5.

  • Havoc 7 hours ago

    Yeah, built on AM4 and in hindsight spending more on mobo & CPU to hop on AM5 would have been the smart move. Live & learn.

    On the plus side I have a lot of non-ECC DDR4 sticks that I'm dumping into the expensive market rn

master_crab 2 hours ago

Don’t do K8s on Pis. The Pis will spend the majority of their horsepower running etcd, CNI of choice, other essential services (MetalLB, envoy, etc). You’ll be left with a minimal percentage of resources for the pods that actually do things you need outside the cluster.

And don’t get me started on if you intend to run any storage solutions like Rook-Ceph on cluster.

reeredfdfdf an hour ago

Maybe I just got lucky, but a year ago or so I managed to find Kingston 32GB DDR4 ECC UDIMM's from Amazon for a price that was more or less identical to normal non-ECC RAM. Running a Ryzen system with 128gb of memory now.