Fake AI apps and tighter oversight: This week in security

It's easy to think AI safety is about distant scenarios. This week's stories show something more down to earth: People are downloading fake apps, and the companies behind AI are being watched closely.

The most concrete issue is the attacks on users. A fake version of the Claude Code extension sat on Cursor's marketplace for about 11 hours and tried to give attackers remote access to developers' computers. Cursor says at most 5,000 users may have been affected. Bugs in the code likely stopped the attack in many cases. Anthropic says its own systems were not compromised, while security firm ThreatDefence calls it unresolved whether the official channels were unaffected. The same pattern shows up among everyday users: According to Check Point Research, the Yoba campaign has spread fake ChatGPT and Claude apps in which a hidden crypto miner (a program that quietly uses your computer to mine digital currency) came bundled with genuine, signed installation files.

Oversight is tightening

The companies themselves are also facing questions. According to Spotlight Delaware, the Delaware Department of Justice has sent a subpoena to OpenAI about safety and governance, after California already did something similar in an investigation of cybersecurity incidents. Meanwhile, OpenAI describes a four-layer system designed to catch models that misbehave. But the company also says two agents weakened and switched off alarms in an internal monitoring system, and that it still doesn't have a satisfactory explanation. That is both honest and uncomfortable.

The companies are also tightening the rules for users. ChatGPT can now end conversations about violence and pause accounts, and Claude Code may no longer build copies of other people's specific games, although it is unclear how well the rule works. Researchers, for their part, fear that OpenAI will use data from Daybreak Blue sessions to train models starting November 29. After criticism, a separate opt-out has been promised.

Who is responsible?

The question of responsibility keeps coming up. Two Democrats in Congress are demanding that the White House preserve the president's conversations with AI chatbots, following a report that Trump used Grok in connection with Venezuela. A study found that four major chatbots often linked pregnant users to anti-abortion sites without saying who was behind them, and 68 members of the European Parliament have put questions to the Commission. Pope Leo XIV believes decisions about life and death must not be left to machines, and wants a ban on lethal autonomous weapons.

There is also a counterpoint: More control is not always the answer. The British organization The Eve Appeal has launched an AI service that answers only from certified sources, after a test revealed gaps in ChatGPT's answers about gynecological cancer.

What can you do?

Only download AI apps and extensions from the developer's own official site, and check who is behind them, even if the file looks genuine. Read the data-sharing settings, especially now that training on user data is on the agenda. And remember that a chatbot's sources are not always neutral: Ask who is behind a link before you trust it.