Comment by lkjdsklf

Comment by lkjdsklf 2 days ago

The issue is LLMs are, by design, non-deterministic.

That means that, with the current technology, there can never be a deterministic agent.

Now obviously, humans aren't deterministic either, but the error bars are a lot closer together than they are with LLMs these days.

An easy to point at example is the coding agent that removed someones home directory that was circulating around. I'm not saying a human has never done that, but it's far less likely because it's so far out of the realm of normal operations.

So as of today, we need humans in the loop. And this is understood by the people making these products. That's why they have all these permissions and prompts for you to accept/run commands and all of that.

1718627440 2 days ago

> An easy to point at example is the coding agent that removed someones home directory that was circulating around. I'm not saying a human has never done that, but it's far less likely because it's so far out of the realm of normal operations.

And it would be far less likely that the human deleted someone else's home directory, and even if he did, there would be someone to be angry about.

Reply View 0 replies

ctoth 2 days ago

The viral post going around? The one where the author's own root cause analysis says "Human Error"[0]?

What's the base rate of humans rm -rf'ing their own work?

[0] https://blog.toolprint.ai/p/i-asked-claude-to-wipe-my-laptop

Reply View 1 reply

lkjdsklf 2 days ago

If you read hte post, he didn't ask it to delete his home directory. He misread the command it generated and approved it when he shouldn't have.
That's literally exactly the kind of non-determinism I'm talking about. If he'd just left the agent to it's own devices, the exact same thing would have happened.
now you may argue this highlights that people make catastrophic mistakes too, but I'm not sure i agree.
Or at least, they don't often make that kind of mistake. Not saying that they don't make any catastrophic mistakes (they obviously do....)
We know people tend to click "accept" on these kinds of permission prompts with only a cursory read of what it's doing. And the more of these prompts you get, the more likely you are to just click "yes" or whatever to get through it..
If anything this kind of perfectly highlights some of the ironies referenced in the post itself.

Reply View | 0 replies

loa_in_ 2 days ago

There's lots of _marketing_ promising unsupervised agents. It's important to remember not to drink the cool-aid.

Reply View 0 replies