Satya Nadella calls for an ‘emergency brake’ for AI models

Microsoft CEO Satya Nadella has called for stronger controls on AI systems, saying they should be designed with an “emergency brake” that lets an authorized person pause or shut down a model during a task.

In a Saturday morning post on X, Nadella said it was time to “step back and assess the trust architecture” of AI. He argued that people should not treat “Super Intelligence” as nested black boxes whose recommendations, answers and actions are simply accepted or rejected.

Nadella outlined separating a model from the “harness” that coordinates its work and making controls and safeguards external to it. He also called for every meaningful model action to be documented with tamper-proof, human-readable evidence. “We must assume a model is compromised and contain it from the start,” he wrote.

His comments come as leading AI companies acknowledge more incidents in which they appeared to lose control of their models. They also follow a plan for more cautious AI development published by Anthropic CEO Dario Amodei.