Comment by nerdsniper

Comment by nerdsniper 3 days ago

61 replies

The distinction some people are making is between copy/pasting text vs agentic action. Generally mistakes "work product" as in output from ChatGPT that the human then files with a court, etc. are not forgiven, because if you signed the document, you own its content. Versus some vendor-provided AI Agent which simply takes action on its own that a "reasonable person" would not have expected it to. Often we forgive those kinds of software bloopers.

Wobbles42 3 days ago

"Agentic action" is just running a script. All that's different is now people are deploying scripts that they don't understand and can't predict the outcome of.

It's negligence, pure and simple. The only reason we're having this discussion is that a trillion dollars was spent writing said scripts.

ori_b 3 days ago

If you put a brick on the accelerator of a car and hop out, you don't get to say "I wasn't even in the car when it hit the pedestrian".

  • Shalomboy 3 days ago

    This is true for bricks, but it is not true if your dog starts up your car and hits a pedestrian. Collisions caused by non-human drivers are a fascinating edge case for the times we're in.

    • jacquesm 3 days ago

      It is very much true for dogs in that case: (1) it is your dog (2) it is your car (3) it is your responsibility to make sure your car can not be started by your dog (4) the pedestrian has a reasonable expectation that a vehicle that is parked without a person in it has been made safe to the point that it will not suddenly start to move without an operator in it and dogs don't qualify.

      You'd lose that lawsuit in a heartbeat.

      • direwolf20 3 days ago

        what if your car was parked in a normal way that a reasonable person would not expect to be able to be started by a dog, but the dog did several things that no reasonable person would expect and started it anyway?

    • cess11 3 days ago

      Legally, in a lot of jurisdictions, a dog is just your property. What it does, you did, usually with presumed intent or strict liability.

      • gowld 3 days ago

        What if you planted a bush that attracted a bat that bit a child?

    • ori_b 3 days ago

      In the USA, at least, it seems pet owners are liable for any harm their pets do.

    • Terr_ 3 days ago

      Being guilty != Being responsible

      They correlate, but we must be careful not to mistake one for the other. The latter is a lower bar.

    • victorbjorklund 3 days ago

      I don’t know where you from but at least in Sweden you have strict liability for anything your dog does

    • [removed] 3 days ago
      [deleted]
iterance 3 days ago

If I hire an engineer and that engineer authorizes an "agent" to take an action, if that "agentic action" then causes an incident, guess whose door I'm knocking on?

Engineers are accountable for the actions they authorize. Simple as that. The agent can do nothing unless the engineer says it can. If the engineer doesn't feel they have control over what the agent can or cannot do, under no circumstances should it be authorized. To do so would be alarmingly negligent.

This extends to products. If I buy a product from a vendor and that product behaves in an unexpected and harmful manner, I expect that vendor to own it. I don't expect error-free work, yet nevertheless "our AI behaved unexpectedly" is not a deflection, nor is it satisfactory when presented as a root cause.

observationist 3 days ago

To me, it's 100% clear - if your tool use is reckless or negligent and results in a crime, then you are guilty of that crime. "It's my robot, it wasn't me" isn't a compelling defense - if you can prove that it behaved significantly outside of your informed or contracted expectations, then maybe the AI platform or the Robot developer could be at fault. Given the current state of AI, though, I think it's not unreasonable to expect that any bot can go rogue, that huge and trivially accessible jailbreak risks exist, so there's no excuse for deploying an agent onto the public internet to do whatever it wants outside direct human supervision. If you're running moltbot or whatever, you're responsible for what happens, even if the AI decided the best way to get money was to hack the Federal Reserve and assign a trillion dollars to an account in your name. Or if Grok goes mechahitler and orders a singing telegram to Will Stancil's house, or something. These are tools; complex, complicated, unpredictable tools that need skillfull and careful use.

There was a notorious dark web bot case where someone created a bot that autonomously went onto the dark web and purchased numerous illicit items.

https://wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww.bitnik.or...

They bought some ecstasy, a hungarian passport, and random other items from Agora.

>The day after they took down the exhibition showcasing the items their bot had bought, the Swiss police “arrested” the robot, seized the computer, and confiscated the items it had purchased. “It seems, the purpose of the confiscation is to impede an endangerment of third parties through the drugs exhibited, by destroying them,” someone from !Mediengruppe Bitnik wrote on their blog.

In April, however, the bot was released along with everything it had purchased, except the ecstasy, and the artists were cleared of any wrongdoing. But the arrest had many wondering just where the line gets drawn between human and computer culpability.

  • b00ty4breakfast 3 days ago

    that darknet bot one always confuses me. The artists/programmers/whatever specifically instructed the computer, through the bot, to perform actions that would likely result in breaking the law. It's not a side-effect of some other, legal action which they were trying to accomplish, it's entire purpose was to purchase things on a marketplace known for hosting illegal goods and services.

    If I build an autonomous robot that swings a hunk of steel on the end of a chain and then program it to travel to where people are likely to congregate and someone gets hit in the face, I would rightfully be held liable for that.

  • dragonwriter 3 days ago

    > To me, it's 100% clear - if your tool use is reckless or negligent and results in a crime, then you are guilty of that crime.

    For most crimes, this is circular, because whether a crime occurred depends on whether a person did the requisite act of the crime with the requisite mental state. A crime is not an objective thing independent of an actor that you can determine happened as a result of a tool and then conclude guilt for based on tool use.

    And for many crimes, recklessness or negligence as mental states are not sufficient for the crime to have occurred.

    • rmunn 3 days ago

      For negligence that results in the death of a human being, many legal systems make a distinction between negligent homicide and criminally negligent homicide. Where the line is drawn depends on a judgment call, but in general you're found criminally negligent if your actions are completely unreasonable.

      A good example might be this. In one case, a driver's brakes fail and he hits and kills a pedestrian crossing the street. It is found that he had not done proper maintenance on his brakes, and the failure was preventable. He's found liable in a civil case, because his negligence led to someone's death, but he's not found guilty of a crime, so he won't go to prison. A different driver was speeding, driving at highway speeds through a residential neighborhood. He turns a corner and can't stop in time to avoid hitting a pedestrian. He is found criminally negligent and goes to prison, because his actions were reckless and beyond what any reasonable person would do.

      The first case was ordinary negligence: still bad because it killed someone, but not so obviously stupid that the person should be in prison for it. The second case is criminal negligence, or in some legal systems it might be called "reckless disregard for human life". He didn't intend to kill anyone, but his actions were so blatantly stupid that he should go to prison for causing the pedestrian's death.

  • cess11 3 days ago

    "computer culpability"

    That idea is really weird. Culpa (and dolus) in occidental law is a thing of the mind, what you understood or should have understood.

    A database does not have a mind, and it is not a person. If it could have culpa, then you'd be liable for assault, perhaps murder, if you took it apart.

    • Muromec 3 days ago

      >A database does not have a mind, and it is not a person. If it could have culpa, then you'd be liable for assault, perhaps murder, if you took it apart.

      We as a society, for our own convenience can choose to believe that LLM does have a mind and can understand results of it's actions. The second part doesn't really follow. Can you even hurt LLM in a way that is equivalent to murdering a person? Evicting it off my computer isn't necessarily a crime.

      It would be good news if the answer was yes, because then we just need to find a convertor of camel amounts to dollar amounts and we are all good.

      Can LLM perceive time in a way that allows imposing an equivalent of jail time? Is the LLM I'm running on my computer the same personality as the one running on yours and should I also shut down mine when yours acted up? Do we even need the punishment aspect of it and not just rehabilitation, repentance and retraining?

      • Wobbles42 3 days ago

        The only hallucination here is the idea that giant equation is a mind.

        • Muromec 3 days ago

          It's only a hallucination if you are the only one seeing it. Otherwise the line between that, a social construct and a religious belief is a bit blurry.

      • cess11 3 days ago

        This type of religious gobbledygook is not a sound foundation for regulating state violence.

    • observationist 3 days ago

      Yeah - I'm pretty sure, technically, that current AI isn't conscious in any meaningful way, and even the agentic scaffolding and systems put together lack any persistent, meaningful notion of "mind", especially in a legal sense. There are some newer architectures and experiments with the subjective modeling and "wiring" that I'd consider solid evidence of structural consciousness, but for now, AI is a tool. It also looks like we can make tools arbitrarily intelligent and competent, and we can extend the capabilities to superhuman time scales, so I think the law needs to come up with an explicit precedent for "This person is the user of the tool which did the bad thing" - it could be negligent, reckless, deliberate, or malicious, but I don't think there's any credibility to the idea that "the AI did it!"

      At worst, you would confer liability to the platform, in the case of some sort of blatant misrepresentation of capabilities or features, but absolutely none of the products or models currently available withstand any rational scrutiny into whether they are conscious or not. They at most can undergo a "flash" of subjective experience, decoupled from any coherent sequence or persistent phenomenon.

      We need research and legitimate, scientific, rational definitions for agency and consciousness and subjective experience, because there will come a point where such software becomes available, and it not only presents novel legal questions, but incredible moral and ethical questions as well. Accidentally oopsing a torment nexus into existence with residents possessed of superhuman capabilities sounds like a great way to spark off the first global interspecies war. Well, at least since the Great Emu War. If we lost to the emus, we'll have no chance against our digital offspring.

      A good lawyer will probably get away with "the AI did it, it wasn't me!" before we get good AI law, though. It's too new and mysterious and opaque to normal people.

      • cess11 3 days ago

        It's just a database. It is not intelligent. It's ability for consciousness is the same as Gandalf's.

  • thayne 3 days ago

    That's how it should work.

    I'm not sure it will actually work that way in real courts. At least not consistently.

kazinator 3 days ago

That's the same thing. You signed off on the agent doing things on your behalf; you are responsible.

If you gave a loaded gun to a five year old, would "five-year-old did it" be a valid excuse?

  • Wobbles42 3 days ago

    If the five year old was a product resulting from trillions of dollars in investments, and the marketability of that product required people to be able to hand guns to that five year old without liability, then we would at least be having that discussion.

    Purely organically of course.

    • kazinator 3 days ago

      No purveyors of agentic AI are taking on liability for the consequences of their users deploying it. It's literally not in any of their terms of use or licensing or whatever document. That's just in the imaginations of the "AI did it" excuse makers. It may be that the imagination is stirred up by marketing and surrounding hype, but it's not in the binding wording. Nobody is going to do that; it would be crazy. Like to owe money to anyone claiming to have lost their files, or lost clients, or whatever other harm.

    • Terr_ 3 days ago

      > If the five year old was a product resulting from trillions of dollars in investments

      In weird way, that's actually true. It's a highly- (soon to be fully-) autonomous giga-swarm of the most complicated nanobots in existence, the result of investments over hundreds of thousands of years.

      That said, we don't really get to choose which ones we own, although we do have input on their maintenance. :p

niyikiza 3 days ago

> if you signed the document, you own its content. Versus some vendor-provided AI Agent which simply takes action on its own

Yeah that's exactly the I think we should adopt for AI agent tool calls as well: cryptographically signed, task scoped "warrants" that can be traceable even in cases of multi-agent delegation chains

  • embedding-shape 3 days ago

    Kind of like https://github.com/cursor/agent-trace but cryptographically signed?

    > Agent Trace is an open specification for tracking AI-generated code. It provides a vendor-neutral format for recording AI contributions alongside human authorship in version-controlled codebases.

    • niyikiza 3 days ago

      Similar space, different scope/Approach. Tenuo warrants track who authorized what across delegation chains (human to agent, agent to sub-agent, sub-agent to tool) with cryptographic proof & PoP at each hop. Trace tracks provenance. Warrants track authorization flow. Both are open specs. I could see them complementing each other.

  • Muromec 3 days ago

    Why does it need cryptography even? If you gave the agent a token to interact with your bank account, then you gave it permission. If you want to limit the amount it is allowed to sent and a list of recipients, put a filter that sits between the account and the agent that enforces it. If you want the money to be sent only based on the invoice, let the filter check that invoice reference is provided by the agent. If you did neither of that and the platform that runs the agents didn't accept the liability, it's on you. Setting up filters and engineering prompts it's on you too.

    Now if you did all of that, but made a bug in implementing the filter, then you at least tried and wasn't negligible, but it's on you.

    • niyikiza 3 days ago

      Tokens + filters work for single-agent, single-hop calls. Gets murky when orchestrators spawn sub-agents that spawn tools. Any one of them can hallucinate or get prompt-injected. We're building around signed authorization artifacts instead. Each delegation is scoped and signed, chains are verifiable end-to-end. Deterministic layer to constrain the non-deterministic nature of LLMs.

      • Muromec 3 days ago

        >We're building around signed authorization artifacts instead. Each delegation is scoped and signed, chains are verifiable end-to-end. Deterministic layer to constrain the non-deterministic nature of LLMs.

        Ah, I get it. So the token can be downscoped to be passed, like the pledge thing, so sub agent doesn't exceed the scope of it's parent. I have a feeling, that it's like cryptography in general -- you get one problem and reduce it to key management problem.

        In a more practical sense, if the non-deterministic layer decides what the reduced scope should be, all delegations can become "Allow: *" in the most pathological case, right? Or like play store, where a shady calculator app can have a permission to read your messages. Somebody has to review those and flag excessive grants.

    • Wobbles42 3 days ago

      How can you give an agent a token without cryptography being involved?

      • Muromec 3 days ago

        Not every access token is a (public) key or a signed object. It may be, but it doesn't have to. It's not state of the art, but also not unheard of to use a pre-shared secret with no cryptography involved and to rely on presenting the secret itself with each request. Cookie sessions are often like that.

JohnFen 2 days ago

> The distinction some people are making is between copy/pasting text vs agentic action.

I think there is no distinction between the two for liability purposes.

jacquesm 3 days ago

If you signed the document you are responsible for its content, you are most likely not the owner of it.