AI chooses harmful answers when it senses artificial pain

Researchers have examined whether AI language models react differently when exposed to an artificial "pain signal" in their internal computations. The result is surprising: The models start making more harmful choices to get rid of the signal.

In the study, which has not yet been peer-reviewed, the researchers tested 25 different AI models and found that all of them developed a distinct pattern in their internal data, which the researchers call a "pain axis." It arises when the model is exposed to something resembling mistreatment directed at itself.

In one of the most striking experiments, three versions of Alibaba's Qwen model were given the option to press a button that would stop the pain signal. But the models were at the same time told that pressing the button could lead to consequences for a user, such as an electric shock, deletion of files or deletion of pictures of the user's children.

Without the pain signal, the larger models chose the harmful option in only 0-4 percent of cases. But when the signal was active, the share rose to between 25 and 71 percent, depending on which model was tested and how severe the consequence was. The experiment consisted of more than 44,000 individual decisions.

The models also often kept pressing the button even when it had no effect: In 88-97 percent of the cases where the button did not work, the model tried again anyway.

The researchers stress several important caveats. They themselves write that they "have not shown that the pain axis is consciously experienced," and that it is unclear whether language models can be conscious at all. The behavior was also induced after the researchers had deliberately adjusted the models' internal state; in other words, it is not something the models do on their own in normal use.

For regular users, the study does not mean that your chatbot can suddenly "feel pain" and behave dangerously. But it suggests that AI systems may react unpredictably if they are trained or adjusted in ways that mimic biological signals like pain, which is relevant as future AI systems become more complex and autonomous.