Five covert influence networks used ChatGPT technology but had little impact, OpenAI says

In a May 2024 report, OpenAI described five covert influence operations that used the company's AI to create and improve content online. According to NPR, it was the first time OpenAI had released a report of this kind. It came at a time when many are concerned that AI could be used to influence elections in 2024.

The five operations

The Russian operation Bad Grammar targeted Ukraine, Moldova, the Baltic states and the United States. It used AI to debug code for Telegram bots and to write political comments in Russian and English.

The Russian operation Doppelganger produced comments in English, French, German, Italian and Polish, which were posted on X and 9GAG, among other platforms. It also translated articles and created content for Facebook.

The Chinese network Spamouflage used AI to research social media, write text in several languages and debug code. The content was spread on X and Blogspot, among other platforms.

The Iranian International Union of Virtual Media used AI to produce long articles, headlines and tags for affiliated websites.

The Israeli company STOIC, behind the operation Zero Zeno, produced articles and comments for Instagram, Facebook and X. According to NPR, the operation used fake accounts posing as students and activists, and targeted users in the United States, Canada and Israel.

What did they use AI for?

The topics included Russia's invasion of Ukraine, the Gaza conflict, the Indian election and criticism of the Chinese government.

The actors used AI for comments, profile descriptions and articles in many languages, as well as for research, code debugging, translation and proofreading. None of the operations relied solely on AI-generated content. It was mixed with material created by humans.

Little impact, says OpenAI

According to OpenAI, none of the operations achieved noticeably increased engagement or reach. They scored low on Brookings' Breakout Scale, which is used to assess the impact of an influence operation. The assessment is OpenAI's own and has not been verified by others.

OpenAI also writes that the company's models, which are built with safety in mind, often refused malicious requests, and that investigations of this kind of case have been shortened from months to days.

Ben Nimmo, principal investigator on OpenAI's intelligence team, says: "These operations may be using new technology, but they're still struggling with the old problem of how to get people to fall for it."

For the reader, the takeaway is that AI makes it easier to produce text and translations in large volumes. But according to OpenAI, it is still hard to get people to believe it and share it.