Comment by abeppu

I encourage you to read the Anthropic blog post, which includes situations like:

- hallucinating conversations with fictitious people

- role playing as a real person and telling them they are present in meatspace, or claimed to have visited a fictional location

- hallucinating details of an account involved in payments

- both ignoring lucrative opportunities and accepting loss-making deals

- making commitments before having done any research

While yes, the skills to run a vending machine and lead a company are not exactly the same, I think the nature of failures discussed means they would likely affect both roles.

I think there's every possibility that a present-day AI allowed to act as CEO would make mediocre choices for some period and then decide that it was a character in a scifi novel selling bespoke space-yachts to comet-mining magnates.