Comment by scottlamb

Comment by scottlamb 3 days ago

4 replies

> How do you store ~100 TB of content with room to grow without a wide NAS?

In the cloud (S3) or on offline (unpowered HDDs or tapes or optical media) I suppose. Most people just don't store that much content.

> So far none of this seems to solve the problem that is solved quite elegantly by the 1U enterprise box... if only you don't look at the power bill.

What kind of power bill are you talking about? I'd expect the drives to be about 10W each steady state (more when spinning up), so 180W. I'd expect a lower-power motherboard/CPU running near idle to be another 40W (or less). If you have a 90% efficient PSU, then maybe 250W in total.

If you're way more than that, you can probably swap out the old enterprisey motherboard/RAM/CPU/PSU for something more modern and do a lot better. Maybe in the same case.

I'm learning 1U is pretty unpleasant though. E.g. I tried an ASRock B650M-HDV/M.2 in a Supermicro CSE-813M. A standard IO panel is higher than 1U. If I remove the IO panel, the motherboard does fit...but the VRM heatsink also was high enough that the top case bows a bit when I put it on. I guess you can get smaller third party VRM heat sinks, but that's another thing to deal with. The CPU cooler options are limited (the Dynatron A42 works, but it's loud when the CPU draws a lot of power). 40mm case fans are also quite loud to move the required airflow. You can buy noctuas or whatever, but they won't really keep it cool. The ones that actually do spin very fast and so are very loud. You must have noticed this too, although maybe you have a spot for the machine where you don't hear the noise all the time.

I'm trying 2U now. I bought and am currently setting up an Innovision AS252-A06 chassis: 8 3.5" hot swap bays, 2U, 520mm depth. (Of course you can have a lot more drives if you go to 2.5" drives, give up hot swap, and/or have room for a deeper chassis.) Less worry about if stuff will fit, more room for airflow without noise.

master_crab 2 days ago

2U is definitely better, but I didn’t notice significant drops in dB till I could stuff a 120mm fan in the case. That requires a 3U or more.

And if you need a good fan that’s quiet enough for the CPU, you’re looking at 4U. Otherwise, you’ll need AIOs hooked up to the aforementioned 120s.

  • scottlamb 2 days ago

    > And if you need a good fan that’s quiet enough for the CPU, you’re looking at 4U.

    Depends on the CPU, I imagine. I'm using one with a 65W TDP. I'm hopeful that I can cool that quietly with air in 2U, without having to nerf it with lower BIOS settings. Many NASs have even lower power CPUs like the Intel N97 and friends.

    • master_crab 2 days ago

      Oh yes, you can definitely get away with much less for something like that or an ARM, Ryzen embedded chips, etc. The 4U is more for full scale desktop CPUs like the i9-12900k I am running (like an NH-D15 sink/fan). You may even be able to get away with passive cooling at the 65W range.

      • scottlamb 15 hours ago

        > You may even be able to get away with passive cooling at the 65W range.

        I saw there's a "passive" Dynatron A43, which even claims to handle up to 155W. My understanding is that most/all server motherboards will have the socket oriented so the fins are front-to-back and the RAM is off to the side. And then you have chassis fans blowing air front-to-back, so I think they basically double as the CPU fan. (Which is also what the older motherboard that came in my CSE-813M did.) I air-quoted passive because I think it needs those chassis fans, but there's not one on the CPU anyway. And I'm not sure I completely trust the A43's rating, but with this setup I think it'd be fine for my 65 W TDP CPU at least.

        On the other hand, I'm using a cheap gaming motherboard with fins sideways, RAM blocking the front-to-back airflow. My gut says that Dynatron A43 wouldn't do well. I don't understand why this orientation is desirable for desktops; my conspiracy theorist side says they make the consumers ones this way so they won't eat into the rack-mounted server market share. I am kinda tempted to get a server motherboard for this and IPMI (and/or at least serial port-accessible BIOS), but I started by looking at budget NASes and things have already spiraled a bit from there.