Comment by adrian_b

Comment by adrian_b 2 days ago

0 replies

If you want instant access to any bit of the 100 TB content, you need a wide NAS.

Otherwise, you can have a couple of HDD racks in which you can insert HDDs when needed (SATA allows live insertion and extraction, like USB).

Then you have an unlimited amount of offline storage, which can be accessed in a minute by swapping HDDs. You can keep an index of all files stored offline on the SSD of your PC, for easy searches without access to HDDs. The index should have all relevant metadata, including content hashes, for file integrity verification and for duplicate files identification.

Having 2 HDD racks instead of just 1 allows direct copies between HDDs and doubles the capacity accessible without swapping HDDs. Adding more than 2 adds little benefit. Moreover, some otherwise suitable MBs have only 2 SATA connectors.

Or else you can use an LTO drive, which is a very steep initial investment, but its cost is recovered after a few hundred TB by the much cheaper magnetic tapes.

Tapes have a worse access time, of the order of one minute after tape insertion, but they have much higher sequential transfer speeds than cheap SATA HDDs. Thus for retrieving big archive files or movies they save time. Transfers from magnetic tape must be done either directly to an NVME SSD or to an NVME SSD through Ethernet of 10 Gb/s or faster, otherwise their intrinsic transfer speed will not be reached.