> LLMs are humanity's "first contact" with non-animal intelligence.
I'd say corporations are also a form of non-animal intelligence, so it's not exactly first contact. In some ways, LLMs are less alien, in other ways they're more.
There is an alignment problem for both of them. Perhaps the lesson to be drawn from the older alien intelligence is that the most impactful aspect of alignment is how the AI's benefits for humanity are distributed and how they impact politics.
I disagree. Corporations are a form of a collective intelligence, or group-think, which is decidedly biological or "animal", just like herding, flocking, hive-minds, etc.
Couldn't the regulations just be viewed as environmental changes that the entities would need to adapt to?
I don't agree entirely, but I do think that "corporation" is a decent proxy of what you can expect from a moderately powerful AI.
It's not going to be "a smart human". It's closer to "an entire office tower worth of competence, capability and attention".
Unlike human corporations, an AI may not be plagued by all the "corporate rot" symptoms - degenerate corporate culture, office politics, CYA, self-interest and lack of fucks given at every level. Currently, those internal issues are what keeps many powerful corporations in check.
This makes existing corporations safer than they would otherwise be. If all of those inefficiencies were streamlined away? Oh boy.
Not really.
Human flaws permeate the entire body of a corporation. The scale may average out some of it, but humans are not just randomly flawed - they're also systematically flawed on the top of it, and averaging does little to counter that.
And the "top end" of the corporation doesn't have enough averaging to mitigate even the random flaws. If someone in the C-suite makes the dumbest decisions ever, the entire system may suffer immense damage before it corrects - if it ever does.
Also: democracy, capitalism/the global economy, your HOA, a tribe, etc etc
Even a weather system is a kind of computational process and “intelligent” in a way
Rob Miles: Think of AGI like a corporation?
https://www.youtube.com/watch?v=L5pUA3LsEaw