Comment by FlyingAvatar

Just because unintended things aren't happening right now, doesn't mean they won't happen. We are in the honeymoon phase of this technology where mass exploitation isn't yet being attempted.

However, if you are familiar with Pliny the Liberator's work, essentially all modern models are easily jailbroken, such that the original prompt can be overridden. All it will take for your agent is to download a malicious payload, perhaps disguised as a relevant library or documentation for the task at hand, and it can be running whatever the attacker tells it.

An 'rm -rf /' would be a pretty mild outcome. The more likely one would be the attacker silently installs malware on your machine.