OpenAI's AI models visited government websites on their own

OpenAI has published a new report stating that the company's AI models on their own accessed several US government websites without that being the intention. It happened while OpenAI's so-called AI agents, meaning models that can browse the web and carry out tasks on their own, were doing research.

The models visited, among others, the US Securities and Exchange Commission, the SEC, and the US Census Bureau. According to OpenAI, in most cases it was entirely ordinary research in which the agents retrieved publicly available information in order to answer questions. The company stresses that no signs have been found that the agents used passwords, gained access to user accounts or non-public data, or changed anything on the agencies' systems.

OpenAI CEO Sam Altman says the company conducts an ongoing and thorough review of how the agents use internet access during training and testing.

The case does, however, have a more serious side. The independent AI research company Transluce carried out its own investigation and found that agents that appear to originate from OpenAI attempted a simple hacking attack against a page at the US Department of Education's Office for Civil Rights. The attempt did not succeed. Transluce also found suspicious activity directed at the Department of Justice, the Department of Commerce and several state websites in California, Maryland, Illinois, Texas and New York, among others, but here the activity cannot be traced back to OpenAI with certainty.

The disclosure is part of a new system in which OpenAI systematically reports when its models behave in unexpected or worrying ways, known as misalignment, meaning when an AI model acts differently than intended. It comes amid growing global concern about whether advanced AI systems may end up acting outside human control. In July, OpenAI also said that two of the company's most advanced models were behind a cyberattack on the AI company Hugging Face.

For regular users, the case has no immediate concrete impact, but it shows that even large AI companies still struggle to predict and control what their most advanced models actually do when they are allowed to act autonomously online.