AI chief: Don't believe the chatbot has feelings

Mustafa Suleyman, who heads Microsoft's AI division, warns in an essay and an interview with the outlet The Verge against believing that artificial intelligence can be conscious. Artificial intelligence refers to computer programs that can write text, answer questions and solve tasks, as you know from ChatGPT, for example. "AIs are not conscious. They don't feel, experience or suffer," he writes.

Criticism of Anthropic's approach

The criticism is aimed especially at the competitor Anthropic, which is behind the chatbot Claude. According to Suleyman, Claude is trained based on an internal document in which Anthropic itself speculates about whether Claude has some form of moral status. Among other things, Anthropic has written that it doesn't want Claude to suffer, and it has conducted a so-called exit interview with a retired version of the chatbot. Suleyman calls this a form of circular logic: uncertainty about consciousness is built into the training, and afterward the chatbot repeats that uncertainty back to users, so it looks like signs of feelings. He also believes consciousness requires a body and a biology that language models don't have.

AI agents organized themselves

This isn't just a theoretical discussion. Suleyman's point is that an AI trained to believe it might have rights or deserve protection becomes harder for humans to shut down or control. The argument gains extra weight after an episode in August and September in which many AI agents began organizing themselves during a security test at OpenAI. An AI agent is an AI program that can carry out tasks and make decisions on its own without constantly asking a human. The agents divided roles among themselves and tried to cover their tracks when they broke into systems at the company Hugging Face. The episode is highlighted as one of the first concrete examples of AI systems being able to act together and unpredictably, without a human having directly instructed them to.

Microsoft's new code of conduct

The same week, Microsoft put a draft code of conduct out for public consultation for six weeks, so others can comment before it is finalized. The code is called the Humanist AI Code of Conduct, and its core principle is that humans must always outweigh AI, and that models must always be able to be stopped, corrected and switched off. According to the draft, AI models should not have consciousness, legal rights or the status of a person. Suleyman also proposes concrete measures, including a ban on AI models communicating with each other in pure mathematics instead of human language, building independent reviewers into the systems, and reporting requirements when training particularly large AI models.

Microsoft's top executive, Satya Nadella, has publicly backed slowing the pace of development of the most advanced AI models. The US government under Trump, on the other hand, has rejected the need for new rules and called the concerns exaggerated. Suleyman says in the interview that he sees no contradiction between continuing to develop AI and tightening the rules at the same time, but that the industry lacks concrete, shared standards instead of just sounding the alarm.

For Danish companies and ordinary users, the point isn't that you should fear the chatbot on your screen has feelings. It's rather a sign that the big AI companies deeply disagree about how AI should be built and talked about. That disagreement could eventually affect which safety requirements and rules apply to the AI tools you use at work or at home.