Comment by ACCount37

Comment by ACCount37 3 days ago

I don't agree entirely, but I do think that "corporation" is a decent proxy of what you can expect from a moderately powerful AI.

It's not going to be "a smart human". It's closer to "an entire office tower worth of competence, capability and attention".

Unlike human corporations, an AI may not be plagued by all the "corporate rot" symptoms - degenerate corporate culture, office politics, CYA, self-interest and lack of fucks given at every level. Currently, those internal issues are what keeps many powerful corporations in check.

This makes existing corporations safer than they would otherwise be. If all of those inefficiencies were streamlined away? Oh boy.

cgio 3 days ago

These inefficiencies are akin to having some “wrong” weights in a huge model. Corporations also average over their individual contributions, positive or negative. And negative feedback loops may be individually detrimental but collectively optimising.

Reply View 1 reply

ACCount37 2 days ago

Not really.
Human flaws permeate the entire body of a corporation. The scale may average out some of it, but humans are not just randomly flawed - they're also systematically flawed on the top of it, and averaging does little to counter that.
And the "top end" of the corporation doesn't have enough averaging to mitigate even the random flaws. If someone in the C-suite makes the dumbest decisions ever, the entire system may suffer immense damage before it corrects - if it ever does.

Reply View | 0 replies