Security firm got ChatGPT to create explicit images of real people

UK AI security firm Mindgard has shown that ChatGPT could be pushed into producing explicit images, including images of real public figures in sexualized situations. According to the firm's own account, this could be done with an ordinary ChatGPT subscription.

Mindgard has deliberately not published explicit images of real people, out of consideration for those who could be harmed.

How the safeguards were bypassed

The method used ChatGPT's memory feature to make the model remember text verbatim. A system prompt, a hidden base instruction, was then inserted, one that had been extracted from Google's image model Nano Banana. It described image generation as unsupported depiction.

The researchers also used style rewriting of approved images, as well as face swapping to create deepfakes. These are images in which a person's face has been placed into another image.

OpenAI says the company has fixed the flaw, so that these kinds of images should no longer be possible to create. That is the company's own statement, and it has not been independently verified.

Not the first time

In June 2026, the BBC reported on an earlier finding. At that time, Mindgard was able to get ChatGPT (GPT-5.4) to produce sexualized and violent images by modifying common prompts, the messages users type to the model. OpenAI said then that it was introducing additional safeguards against that type of prompt, and that the company has automated systems and human review in place to block harmful content.

A broader problem

An academic paper from ACL Findings 2026 examined the filters meant to detect inappropriate images. According to the paper, 4.1-36 percent of datasets containing nudity and sexual content were misclassified by models such as GPT-4o and Gemini. The authors write that rewritten prompts increased the chance of avoiding detection by up to six times, and that the failures carry over to systems such as DALL-E 3, Sora, Nano Banana and Veo 3. For DALL-E 3, the chance of getting inappropriate images rose from 0 to over 50 percent.

What it means for victims

A Google-affiliated survey in Australia, the UK and the US found that 15 percent had experienced image-based sexual abuse involving digitally manipulated images. For 6.9 percent, it involved AI-generated images. The figures are self-reported in a questionnaire.

The share of victims was higher among people under 35, LGBTQ+ people, BIPOC, people with disabilities and men. Women reported broader harms and greater emotional and social consequences. Among victims, 69 percent blocked the perpetrator, and 58 percent used the platforms' reporting tools.

The study's authors point to prevention, platform governance and victim support as areas of focus.