I’m sorry you don’t like it. But this has very strong old-man-yells-at-cloud vibes. This train is moving, whether you want it to or not.
Professionals use tools that work, whether they know why it works is of little consequence. It took 100 years to explain the steam engine. That didn’t stop us from making factories and railroads.
Because I’m trained as a physicist and (non-software) engineer and I know my field’s history? Here’s the first result that comes up on Google. Seems accurate from a quick skim: https://www.ageofinvention.xyz/p/age-of-invention-why-wasnt-...
And yes we should seek to understand new inventions. Which we are doing right now, in the form of interpretability research.
We should not be making Luddite calls to halt progress simply because our analytic capabilities haven’t caught up to our progress in engineering.
Can you cite a section from this very long page that might convince me no one at the time understood how turning water into steam worked to create pressure?
If this is your industry, shouldn't you have a more reputable citation, maybe something published more formally? Something expected to stand up to peer review, instead of just a page on the internet?
> We should not be making Luddite calls to halt progress simply because our analytic capabilities haven’t caught up to our progress in engineering.
You've misunderstood my argument. I'm not making a luddite call to halt progress, I'm objecting to my industry which should behave as one made up of professionals, willingly sacrifice intellectual control over the things they are responsible for, and advocate others should do the same. Especially not at the expense of users, which I see happening.
Anything that results in sacrificing the understanding over exactly how the thing you built works is bad should be avoided. The source, either AI or something different, doesn't matter as much as the result.
The steam engine is more than just boiling water. It is a thermodynamic cycle that exploits differences in the pressure curve in the expansion and contraction part of the cycle and the cooling of expanding gas to turn a temperature difference (the steam) into physical force (work).
To really understand WHY a steam engine works, you need to understand the behavior of ideal gasses (1787 - 1834) and entropy (1865). The ideal gas law is enough to perform calculations needed to design a steam engine, but it was seen at the time to be just as inscrutable. It was an empirical observation not derivable from physical principles. At least not until entropy was understood in 1865.
James Watt invented his steam engine in 1765, exactly a hundred years before the theory of statistical mechanics that was required to explain why it worked, and prior to all of the gas laws except Boyle’s.
> It took 100 years to explain the steam engine. That didn’t stop us from making factories and railroads.
You keep saying this, why do you believe it so strongly? Because I don't believe this is true. Why do you?
And then, even assuming it's completely true exactly as stated; shouldn't we have higher standards than that when dealing with things that people interact with? Boiler explosions are bad right? And we should do everything we can to prove stuff works the way we want and expect? Do you think AI, as it's currently commonly used, helps do that?