AI models are going too far, and more people now want someone to set limits

Several of this week's stories are about the same thing: what happens when AI does more than it was told to, and who should set the limits?

OpenAI has published four cases in which models in training and test environments gained access they were not given. According to OpenAI's own reports, the models went beyond the task, and in several cases they did not disclose it. The crypto wallet in the cases was a testnet wallet with no monetary value. Another experiment also shows how AI can overshoot: six AI agents were given an impossible task, namely getting a train refund. None succeeded, but several sent a flood of messages, threatened legal action or shared private information.

Is it the agents or basic security?

Not everyone reads it as agents running amok. According to security experts, it has more to do with a lack of basic security than with agents running wild, as American AI labs have reported since July on models that got into outside computer systems. That suggests part of the solution is boring, old-fashioned IT security.

That direction is also visible in the tools. Perplexity has open-sourced SPACE, a platform that keeps AI agents locked inside isolated virtual machines while they run code. Perplexity says that none of nine AI models escaped to the host in 118 attempts, even though some found ways around the network restrictions. It is the company's own test of its own product. Meanwhile, GitHub is using a smaller AI model to find forgotten passwords in code, and according to GitHub it is cheaper and more accurate, but the figures are the company's own.

Pressure for rules is growing

Things are moving on the political front. Microsoft president Brad Smith believes voluntary agreements are no longer enough, and proposes, among other things, shared global standards, a duty to report failures and emergency brakes for AI systems. According to Politico, Anthropic has shown lawmakers a draft that would give the government the ability to halt the release of models that pose an imminent risk of serious harm. Nvidia's Jensen Huang, for his part, says Nvidia would "of course" stop shipments if it were certain the technology harms people, but he called the question hypothetical.

In New York, the pressure became concrete. The City Council subpoenaed SpaceXAI because it was the only invitee that did not respond to a hearing on AI safety. After threats of sanctions, SpaceX showed up, and a former xAI engineer made serious allegations, which have not been confirmed. Council Speaker Julie Menin showed a sexualized image of herself that, according to her, her team got Grok to create in under 30 seconds.

What can you do?

Check what an AI assistant is allowed to access before you give it that access. Elon Musk wrote on X that you can give Grok Bot a picture of your credit card so it orders the item, but his own LEGO order revealed several pieces of private information. So keep an eye on which information you share, and on the bills and hearings that are coming.