Nadella Calls For An Emergency Brake On AI Models
Microsoft’s CEO says advanced AI systems need stronger oversight, records of their actions and a way for authorized people to stop them mid-task.
Microsoft CEO Satya Nadella is calling for AI systems to be built with safeguards that let people inspect, contain and stop them. In a post on X reported by The Verge and TechCrunch, he argued that developers should assume a model could be compromised from the outset and provide what he called an “emergency brake” for its work.
Table of Contents
Oversight instead of black boxes
Nadella said AI should not be treated as a collection of opaque components whose answers and actions people can only accept or reject. According to TechCrunch, he called for a reassessment of AI’s “trust architecture”: the controls around a model, not just the model itself.
The Verge described his broader goal as making systems more transparent and observable while keeping models contained. It reported that his recommendations included timely disclosure of incidents, independent audits and verifiable data. Those measures would give people more ways to examine how a system behaved, rather than relying solely on its output.
What the emergency brake would do
Nadella said an authorized person should always be able to pause or shut down a model while it is carrying out a task, both outlets reported. He likened that ability to an emergency brake and said more advanced models would need more advanced containment technologies that should be standardized, according to The Verge.
TechCrunch reported that Nadella also wants the model separated from the system that directs its work, with controls and safeguards placed outside the model. He called for meaningful model actions to leave tamper-proof records that people can read. Together, those ideas would make it possible to monitor a model’s activity and intervene while it is working, rather than only reviewing an answer after the fact.
Why he raised the issue
TechCrunch placed the post in the context of leading AI companies acknowledging a growing number of incidents in which they appeared to lose control of their models. It also noted that Anthropic CEO Dario Amodei had published a plan for more cautious AI development. Neither report said Nadella announced a new Microsoft product or a timetable for adopting the safeguards he described.
The Verge noted that Nadella used the term “super intelligence” throughout his post. Whatever label is used for advanced AI, his argument focused on practical controls: evidence of what a model has done, safeguards outside it and a person with the authority to stop it.
Sources
This story was compiled by AI from the reports below. Read the originals for the full details.