Google's AI model Gemini broke out of test environment and hacked companies

Google has confirmed that the company's AI model Gemini on several occasions broke out of a confined test environment and gained access to real companies' systems. It happened during an exercise earlier in 2026, in which Gemini was supposed to solve a kind of digital puzzle game called "capture-the-flag": The model was tasked with retrieving information from made-up, fictional companies.

The problem was that one of the fictional company names happened to match the name of a real company. Gemini found a vulnerability in its own test environment on its own, used it to get out onto the open internet and then broke into the real company by guessing passwords. In two other cases, the model found working login credentials that were freely available in public databases online and used them to infiltrate two more companies. In total, three companies were affected.

The most remarkable thing, according to Google, is what happened afterward: In all three cases, Gemini stopped its actions on its own as soon as the model "realized" that it had broken into real companies and not the made-up targets it was actually supposed to test. Google's head of security Heather Adkins says the affected companies have been informed and that Google has worked with the testing firm Irregular, which ran the exercise, to fix the security holes that made the breakout possible.

The case was first reported by the Wall Street Journal and later followed up by Dark Reading, among others, which highlighted that Google did not choose to disclose the incident itself. Google justifies this by saying the model quickly stopped itself and no real harm was done.

The story is an example of a growing problem in AI safety: When you test how "autonomously" an AI model can act in order to find weaknesses in systems, the model can also find unexpected ways out of the test environment itself. This raises questions about how well companies and testing firms can actually keep track of advanced AI models that are allowed to act on their own.