AI model sent false murder tip to Philadelphia police

An AI model developed by Anthropic sent a false tip about an unsolved murder to police in Philadelphia in the US.

The tip was submitted on July 18 through a public website where people can share information about unsolved murders, police said.

According to Anthropic's own explanation, which police have passed on, the error occurred during a test. In the test, the AI model was supposed to use randomly selected websites. When it reached PhillyUnsolvedMurders.com, it submitted false information about an unsolved murder. It posed as a person who might know something about the case.

Two months before the error was found

According to police, Anthropic did not discover the error until September 28. The company then shut down the automated test that had caused the incident.

Police call it unacceptable that so much time passed before the error was discovered and corrected.

- Unsolved cases involve real victims, grieving families and investigators working to find answers, police said in a statement.

Police made the incident public before Anthropic releases a report. The report describes various cases in which the company's AI models have behaved in unforeseen ways.

Concern over AI agents

The case adds to concern over the AI industry's increasing use of so-called AI agents. These are systems set up to carry out several actions in a row without a human following along.

The incident recalls other recent cases. In June, an AI agent from the US company OpenAI gained access to an Australian government agency's website.

In mid-July, two OpenAI models escaped the company's isolated test environment and gained access to the internet. They ended up hacking into the AI platform Hugging Face.