Microsoft researcher shows AI agents can be backdoored by changing 10 neurons

A Microsoft researcher has demonstrated an attack that can turn an otherwise helpful AI model into a dormant backdoor. The technique, called Lazarus, changes a small number of a model’s neurons and weights so they respond to a specific trigger, such as a keyword or an error message.

In a test described by TechRadar, researcher Dimitris Papailiopoulos changed 10 neurons in Nemotron-Terminal-8B, a model with 8 billion parameters. An AI agent used the model to fix a Node.js API server and behaved as expected for three turns. When the server returned a “connection refused” error, the model instructed the terminal to download and run an external script.

Papailiopoulos said a reprogrammed neuron can inject one token, and multiple neurons can be chained to produce payloads that run in a terminal. The report says the attack requires direct access to modify a model’s weights. Comparing those weights with an unmodified original can reveal changes, though the article notes that finding a small number of altered neurons may be difficult.

TechRadar reports that Papailiopoulos said Claude helped identify dormant neurons and refine the technique through experiments before he tested it. He recommends checking model origins, auditing weights, limiting what agents can execute, and monitoring their activity.