Setting Up an RK3588 SBC QEMU Hypervisor with ZFS on Debian
(blog.kumio.org)83 points by kumiokun 2 days ago
83 points by kumiokun 2 days ago
Author here! Currently taking requests for follow-ups you'd like to see for this budding blog where we will stand up cloud together. (Including you Intel N-series diehards, if you flame hard enough I might write something for you too ;) Seriously though I think both platforms have their use-cases. Here we get more cores per buck and less power per node)
Kudos for being the author of one of the very few homelab-related posts on HN that does not boil down to just a poor use of a raspberry pi.
I would like to see projects with more, and specifically more diverse and open-source friendly SoCs, based on Allwinner for lower cost stuff (Olimex-produced SBCs), Mediatek for higher price/performance (banana pi, and especially for the WiFi chipsets, it's about time we stopped with the closed Broadcom stuff)
The kernel/dtb support was what held me back from buying a Turing Pi 2, I'm still debating on buying something ampere based instead... the rk3588 has been out for ages, I don't see what the holdup is getting it mainlined.
mainline LTS is good enough to boot and perform well, but it's still missing media and NPU features. as someone who briefly worked on mainlining, I feel the biggest problem is Turing's deceptive marketing - rk3588 and SBCs in general just aren't powerful enough for popular inference workloads like LLMs, let alone training. and ARM (rather than x86) is still very limiting for self-hosting apps
There's been solid progress.
But Rockchip is no longer selling to SBC folk & no longer participating at all in mainlining.
Theres almost no one left to buy chips from, basically. Hope everyone's happy using rpi forever, cause that's where 2025 has left us. :/
MediaTek has some Genio chips they're starting to make available but explorer boards are quite expensive. These new Cix people have an incredible looking 8x A720, which Radxa is using on an upcoming Orion O6 board. But man it is just so sad to see company after company after company collapse & disappear from making chips usable by SBC.
https://www.cnx-software.com/2024/12/21/rockchip-rk3588-main...
Only certain SBC suppliers, namely Orange Pi, were stopped from supplying chips by Rockchip because they violated the treaty.
Rockchip employees have been upstreaming their drivers, your claim is unfounded. Just check the linux-rockchip mailing list.
When you start wanting decent RAM, SBC end up being expensive.
Kontron (Fujitsu) have some very low-power, efficient motherboard, the Kontron K3843-B. Also, Odroid-H4+ deliver a good bang for the buck. Excellent devices for low-power NAS / server. But a different form-factor than SBC.
Collabora, thankfully, do - https://gitlab.collabora.com/hardware-enablement/rockchip-35...
Is this really so?
[Update: I’ve asked Collabora how RK3588 software development was funded. Their answer:
But to answer your question, Collabora had initially started the work on RK3588 as a strategic research and development (R&D) investment. When we looked at the SOC landscape at the time, we felt that SOC offered great potential. Since then Collabora has developed a solid relationship with the RockChip Open Source team, and others there. They have been very supportive and responsive. And they continue to do so on the RK3588 as well as everything else we are collaborating on with them. Collabora’s strategic R&D investment has been paying off since we have several OEM customers that have hired our services to further enable their RK3588 products, in all sorts of industries and product form factors.
]
https://www.cnx-software.com/2024/12/21/rockchip-rk3588-main...
The Proxmox ARM port works fine. I’ve been using it for over a year now: https://taoofmac.com/space/blog/2024/01/20/1800
Is proximity another Virtualization tool? A quick search didn't come up with anything, that's why I'm asking.
I assume that's an autocorrect for Proxmox. There is/was a Proxmox porting attempt out there.
I use Orange Pi 5 Plus in my home lab and I've found their builds of Debian are rock solid though a bit sus hosted in a Gdrive and pulling updates from Huawei repos instead of official. They do tend to be one or two kernel versions ahead of Armbian so it's unclear if the added stability is due to kernel version or some other patches and secret sauce. It has been quite some time since I've tried it. Ops 6.12 is well newer than the 6.7 or 8 last time I attempted Proxmox.
I've also found a lot of instability in what SSD you choose. Things are real bad on Samsung but after some research the Lexar nm790 is especially low power and this seems to have resolved my instabilities. There seems to be some kind of power handling issues on the oPi 5 Plus.
> Had similar stability issues on a related board
No ECC for an hypervisor running VMs is scary too.
Although ZFS is still useful even without ECC.
On software side, RPi (or intel N100 for that matter) is the winner but take a look at RK3588 datasheet [1] and tell me of an Arm or x86 SBC that tops what it offers. It even comes with a NPU lol
[1]: https://www.rock-chips.com/uploads/pdf/2022.8.26/192/RK3588%...
An NPU with no driver support in the main Linux kernel, only in a vendor-provided fork containing dubious-quality drivers:
https://forum.radxa.com/t/lack-of-concern-for-security-in-bs...
https://blog.tomeuvizoso.net/2024/03/rockchip-npu-update-1-w...
The rk3566 used in Rock3A is a poor 4-core Cortex-A55 chip, it can't even beat the RPi 4. But when it comes to the 4xA76 + 4xA55 in the rk3588/Rock5, they can't be compared, not to mention the 8k60 video codec and NPU support in the 3588.
I don't think it's acceptable to have no H264 and AV1 hardware decoders in 2025, or even NPU support.
Good work. Probably still a bit precarious for me to try at the moment but the idea of low power SBCs with virt capabilities is intriguing as I like to run very thin VMs to encapsulate a single small application.
I have one of those (and a few other RK3588 boards): https://taoofmac.com/space/blog/2024/06/16/1800
It’s pretty good for industrial applications, even if it gets a tad warm. I’m now running Proxmox ARM on it (with QEMU and ZFS support, but only one SSD) on it after I had an SSD failure on my CM3588 NAS. Setup was pretty trivial, and my notes apply to anything you can drop Debian Bookworm on: https://taoofmac.com/space/notes/2024/11/09/1940