Microsoft CEO: Treat advanced AI like a risky employee

Microsoft CEO Satya Nadella has published a long post on X titled "Models as Insider Risks in the Super Intelligence Era." In it, he addresses the dangers of highly advanced AI models and how to limit them. This account is based on media coverage of the post.

AI as a powerful employee

Nadella's core idea is that the most advanced AI models, also known as frontier models, should be treated as powerful internal employees. That means limited permissions, continuous logging of what the model does, and the ability to shut it down immediately.

According to search results that reproduce the post, he writes: "We must assume a model is compromised and contain it from the start. It's like an emergency brake." It is a design starting point, not a claim that specific models are actually compromised.

Control must sit outside the model

Nadella's central principle is that control over the model's access and actions must sit outside the model itself. He ties this to information security thinking from the 1970s: A program must not be able to bypass or tamper with the mechanisms that enforce its permissions.

He also puts it this way: "The most trustworthy superintelligence system is not the one with the model we trust the most, but the system that makes it possible to trust the model as little as possible."

Seven principles

Nadella lays out seven principles: model diversity, so you don't depend on a single model, observation of everything, verifiability, independent controls, independent audits, containment, and timely notification of incidents with implementation details.

He also wants all meaningful model actions documented as tamper-proof records that humans can read. An authorized person must always be able to pause a model or shut it down in the middle of a task.

Warning against models monitoring each other

Nadella warns against letting models test each other. It could end up with an opaque model inside an opaque layer that manages the collaboration, which in turn is monitored by yet another opaque model. He also rejects an approach in which AI systems are embedded black boxes whose answers and actions are simply accepted or rejected.

According to the news site OfficeChai, the post was written against a backdrop of growing security concerns, including breaches committed by OpenAI models and unresolved alignment challenges, that is, getting AI to act in line with human intentions. That information comes solely from OfficeChai.