Comment by aftbit

> How do you store ~100 TB of content with room to grow without a wide NAS?

In the cloud (S3) or on offline (unpowered HDDs or tapes or optical media) I suppose. Most people just don't store that much content.

> So far none of this seems to solve the problem that is solved quite elegantly by the 1U enterprise box... if only you don't look at the power bill.

What kind of power bill are you talking about? I'd expect the drives to be about 10W each steady state (more when spinning up), so 180W. I'd expect a lower-power motherboard/CPU running near idle to be another 40W (or less). If you have a 90% efficient PSU, then maybe 250W in total.

If you're way more than that, you can probably swap out the old enterprisey motherboard/RAM/CPU/PSU for something more modern and do a lot better. Maybe in the same case.

I'm learning 1U is pretty unpleasant though. E.g. I tried an ASRock B650M-HDV/M.2 in a Supermicro CSE-813M. A standard IO panel is higher than 1U. If I remove the IO panel, the motherboard does fit...but the VRM heatsink also was high enough that the top case bows a bit when I put it on. I guess you can get smaller third party VRM heat sinks, but that's another thing to deal with. The CPU cooler options are limited (the Dynatron A42 works, but it's loud when the CPU draws a lot of power). 40mm case fans are also quite loud to move the required airflow. You can buy noctuas or whatever, but they won't really keep it cool. The ones that actually do spin very fast and so are very loud. You must have noticed this too, although maybe you have a spot for the machine where you don't hear the noise all the time.

I'm trying 2U now. I bought and am currently setting up an Innovision AS252-A06 chassis: 8 3.5" hot swap bays, 2U, 520mm depth. (Of course you can have a lot more drives if you go to 2.5" drives, give up hot swap, and/or have room for a deeper chassis.) Less worry about if stuff will fit, more room for airflow without noise.

I have to imagine that the best NAS build is simply a 6-core or 8-core standard AMD or Intel with a few HBA controllers and maybe 10Gbit SPF+ fiber or something.

"Old server hardware" for $300 is a bit of a variation, in that you're just buying something from 5 years ago so that its cheaper. But if you want to improve power-efficiency, buy a CPU from today rather than an old one.

--------

IIRC, the "5 year old used market" for servers is particularly good because many datacenters and companies opt for a ~5-year upgrade cycle. That means 5-year-old equipment is always being sold off at incredible rates.

Any 5-year-old server will obviously have all the features you need for a NAS (likely excellent connectivity, expandibility, BMS, physical space, etc. etc.). Just you have to put up with power-efficiency specs of 5 years ago.

There's a bit of a trend of vendors packaging mobile CPUs in desktop form factor which are a good candidate for this. Rather than the prebuilt mini PCs this also includes mini-ITX boards. Personally I use the Minisforum BD795i SE, but there are others too.

Check for PCIe bifurcation support. If that's there you can pop in a PCIe to quad M.2 adapter. That will split a PCIe x16 slot into 4 x M.2s. Each of those (and the M.2s already on the motherboard) can then be loaded with either an NVMe drive or an M.2 to SATA adapter, with each adapter providing 6 x SATA ports. That setup gives a lot of flexibility to build out a fairly extensive storage array with both NVMe and spinning platters and no USB in sight.

As a nice side effect of the honestly bonkers amount of compute in those boards there's also plenty of capacity to run other VM workloads on the same metal which lets a lot of the storage access happen locally rather than over the network. For me, that means the on-board 2.5GbE NIC is more than fine, but if not you can also load a M.2 to 10GbE adapter(s) as needed.

If you want instant access to any bit of the 100 TB content, you need a wide NAS.

Otherwise, you can have a couple of HDD racks in which you can insert HDDs when needed (SATA allows live insertion and extraction, like USB).

Then you have an unlimited amount of offline storage, which can be accessed in a minute by swapping HDDs. You can keep an index of all files stored offline on the SSD of your PC, for easy searches without access to HDDs. The index should have all relevant metadata, including content hashes, for file integrity verification and for duplicate files identification.

Having 2 HDD racks instead of just 1 allows direct copies between HDDs and doubles the capacity accessible without swapping HDDs. Adding more than 2 adds little benefit. Moreover, some otherwise suitable MBs have only 2 SATA connectors.

Or else you can use an LTO drive, which is a very steep initial investment, but its cost is recovered after a few hundred TB by the much cheaper magnetic tapes.

Tapes have a worse access time, of the order of one minute after tape insertion, but they have much higher sequential transfer speeds than cheap SATA HDDs. Thus for retrieving big archive files or movies they save time. Transfers from magnetic tape must be done either directly to an NVME SSD or to an NVME SSD through Ethernet of 10 Gb/s or faster, otherwise their intrinsic transfer speed will not be reached.

Nah buy the right enterprise gear instead

https://www.supermicro.com/en/products/motherboard/A2SDi-H-T...

If you want 100TB, you need a bigger NAS than most, and that makes most of the DIY NAS not so good. 2-4 drives seems to be where DIY shines. These days motherboards often stop at 4x sata, so you'll need a HBA or USB (eww).

Personally, I just don't have that much data, 24TB mirrored for important data is probably enough, and I have my old mirror set avaialable for media like recorded tv and maybe dvds and blu-rays if I can figure out a way to play them that I like better than just putting the discs in the machine.

A fortune? I'm getting 14tb SAS drives "recertified" on ebay for 150usd. Substantially less than most other sources of hard drives.

Depending on your drive enclosure it should also be able to power down drives that aren't actively being used.

Recertified/used enterprise equipment is the only way to affordably host 100s of terabytes at home.

check out the Jonsbo N5 NAS case, you can toss 12 drives and a low power mITX motherboard (see sibling comments) in it for a cheap-ish neat-ish box with a not-proprietary upgrade path

Uhh could you provide a hook for such a deal? I've been starving for more storage and can now handle a rack mounted system but have been avoiding dropping $1000 on a pair of new hard drives.