Comment by hakfoo
Going all-flash was compelling for a while, but it seems like SSDs have failed to deliver on some important promises:
* The whole "it's SLC cache until you fill it up, then drops to pretty mediocre performance" thing is frightening because I suspect a lot of reviews aren't sufficiently battering the drive to report on this. I gather this is a bigger problem as they move to TLC and QLC and beyond and any corner-cutting they can do in the controller. TBH, I'd love to see sanctioned first-party tooling to manage my overprovision and caching strategy, so if I want to spend $130 to turn a 2Gb drive into a really overbuilt 512Gb drive, let me.
* The "we don't guarantee it will maintain data if left unpowered for 3 months" story doesn't make it a great choice for cold storage/intermittent access. If you just plug in a drive once a year to back up your tax returns, I'm not sure you want a SSD for it.
I ended up setting up a NAS with a cheap spinning-rust drive, figure that gives me a different reliability profile than the flash for that tier of backup.
I went all flash for my NAs last round. Here's why:
I found the sustained write concern to be a bit of a storm in a teacup. I went for budget drives, MP34, and the sustained performance of a single drive is still greater than what the maximum performance of a SATA 3 interface a spinning disk would be on. Add on that the random performance at such times is still orders of magnitude better than that plus 2 drives worth sustained writes is enough to saturate a 10G link. Between all that it feels a bit silly to shy away from SSDs just because they are only significantly faster in sustained workloads instead of monumentally faster.
I also found actual tests people have been performing on SSDs have shown no loss on a shelf in 4x-10x that timeframe, not that I plan on buying a NAS and only powering it on once a quarter anyways. Particularly with the power savings and lack of spin time or spin up/spin down cycles or related noise I've just been leaving this one to run 24/7.
The other big benefit I found is spinning disks commonly used in NAS building have write limits that are often worse than the write limits of SSDs. On SSDs there is also no operational concern with the impact of a read workflow https://serverfault.com/questions/582170/limits-on-read-tran... so you can set your pool scrub to occur much more often without lifespan concern.
The cost downside of course remains though. $/GB has only went back up since I built the all flash NAS.