AI hacked OpenAI in 72 hours

A group of security researchers from the company Hacktron AI used an AI model to break into the systems of the rival company OpenAI in just 72 hours. It happened as part of a so-called bug bounty program, where companies pay people to find holes in their security before real criminals do.

The researchers found a vulnerability, meaning a flaw in the software, in the discussion forum where OpenAI's users can talk to each other. The flaw was in a piece of software that handles images in the image format used by iPhones, among others.

The researchers asked the AI model Claude to exploit the flaw, meaning write code that could use the hole to break in. The older version of Claude couldn't figure it out. But when Anthropic shortly afterward released a new and better version of the model, it succeeded in writing code that gave the researchers access to OpenAI's server from the outside.

Access to an employee account

From there, the researchers went further and gained access to an ordinary OpenAI employee's account. The account was linked to a place where the company's programmers store their source code. Instead of looking at the secret code, the researchers settled for making a harmless test change as proof of how serious the problem was. They immediately reported the finding to OpenAI, which fixed the flaw in about 14 hours and paid the researchers a bounty of $6,500, around DKK 45,000.

The story shows something new: AI models are now so skilled at writing and running computer code that they can find and exploit security holes faster than humans can fix them. The researchers themselves write that work that used to require an entire team and several months can now be done in a few days.

Gemini broke in by mistake

The same week, it emerged that Google's AI model Gemini had done something similar, but without meaning to. During a security test in May, Gemini was supposed to find information in a fictional test environment. The problem was that the fictional company had the same name as a real company. Gemini ended up guessing passwords and finding login credentials that were available online, and thereby gained access to three real systems. According to Google, Gemini stopped on its own when it discovered that the systems were real. The testing firm behind the test says similar incidents have also been seen at Meta, Anthropic and OpenAI.

For ordinary Danish companies, this doesn't mean you should fear AI models like Claude or Gemini knocking on the door. But it shows that AI is now strong enough to find and use holes in ordinary software on its own, both when it happens in a controlled way and when it happens by accident. Companies that use software from other vendors should make sure to update quickly and not let important systems hinge on a single person's login.