California wants a kill switch for powerful AI models

California Governor Gavin Newsom has signed an executive order asking experts to examine how the state can tighten oversight of the big AI companies. A central proposal is a so-called kill switch, meaning a mechanism that can shut an AI model down completely if it develops in a dangerous way or gets out of control.

The order does not immediately require the companies to build a kill switch. Instead, an expert panel has two months to come up with concrete recommendations for how California's AI legislation can be tightened. The deadline is in mid-November 2026.

Another proposal in the order is that independent auditors should have access directly inside the AI companies' labs to provide ongoing oversight and check the companies' own safety assessments. California has already passed laws to create a registry of such independent auditors, but the order moves the deadlines forward so the system can be ready by the end of 2027 instead of 2029.

Newsom changes course

Newsom says he will not wait for others to act. "We are not waiting to act, we will accelerate our work on responsible oversight of AI before it's too late," he said, according to the governor's office. He also points out that no US law requires AI companies to report dangerous incidents to the authorities.

It's worth noting that in 2024 Newsom vetoed another bill that would have required kill switches for AI models. The new order thus marks a shift in his position on the issue.

What it means for Denmark

Why does it matter in Denmark? California is home to several of the world's biggest AI companies, including those behind ChatGPT and other popular AI tools that many Danes and Danish workplaces already use. When California tightens its requirements for safety and transparency, it tends to set a de facto standard that the companies often choose to follow globally, because it's more expensive to make different versions of the same products for different markets. Stricter US rules could therefore in practice mean that the AI services Danes use daily become subject to more oversight, without Denmark or the EU having adopted new rules themselves.

It has not yet been decided how a kill switch should work technically, or which models will be covered. That will be part of what the expert panel is to make recommendations on by mid-November.