Comment by zephen
> If calculators returned even 99.9% correct answers, it would be impossible to reliably build even small buildings with them.
I think past successes have led to a category error in the thinking of a lot of people.
For example, the internet, and many constituent parts of the internet, are built on a base of fallible hardware.
But mitigated hardware errors, whether equipment failures, alpha particles, or other, are uncorrelated.
If you had three uncorrelated calculators that each worked 99.99% of the time, and you used them to check each other, you'd be fine.
But three seemingly uncorrelated LLMs? No fucking way.
There's another category error compounding this issue: People think that because past revolutions in technology eventually led to higher living standards after periods of disruption, this one will too. I think this one is the exception for the reasons enumerated by the parent's blog post.