Comment by michaelmrose

Comment by michaelmrose 2 days ago

Shouldn't all agentic actions with meaningful outputs of importance like moving $48,000 simply be required to terminate in a human designed or verified output with a human in the loop attestation.

Eg a list of transactions that isn't AI generated where the only actions that actually move money must operate on the data displayed in the human designed page.

A human looks at this and says yes that is acceptable and becomes reasonable for that action.

krapp 2 days ago

The entire goal of AI is to not have humans in the loop at all.

So while that should happen, it won't. They'll just add an extra layer of AI to do the verification.

Reply View 2 replies

michaelmrose 2 days ago

If AI were good enough to detect hallucinations wouldn't that be built into the AI already?

Reply View | 1 reply
- niyikiza 2 days ago
  
  Exactly ... and that's why I'm skeptical of "AI verifies AI" as the primary safety mechanism. The verifier for moving money should be deterministic: constraints, allowlists, spend limits, invoice/PO matching, etc. The LLM can propose actions, but the execution should be gated by a human/polic-issued scope that's mechanically enforced. That's the whole point: constrain the non-deterministic layer with a deterministic one. [0] [0] https://tenuo.dev/constraints
  
  Reply View | 0 replies