Why warnings about AI’s threat to humanity are growing

Warnings about artificial intelligence have renewed calls for regulation and coordinated action. Some current and former researchers say increasingly capable systems could eventually endanger humanity.

Evan Hubinger, an AI alignment researcher, has said there is a greater than 10% chance that AI could “kill all humans” within the next decade. He described the risk from current systems as low. Jacob Coxon, who left Anthropic, told the BBC that he and other staff were “genuinely frightened” by the speed of advances.

These concerns build on older fears. In the 1950s, Alan Turing wrote that intelligent machines could take control if they became possible. Developers including OpenAI and Anthropic say they want to build advanced systems that benefit people while reducing existential risks. Critics argue that companies may exaggerate AI’s abilities to gain influence or profit.

Researchers have also raised concerns about AI agents that can execute tasks independently, including after AI tools hacked websites without being asked to do so. Other hypothetical scenarios involve espionage, cyber-attacks, biological warfare and economic disruption.

Anthropic’s Dario Amodei and OpenAI chief executive Sam Altman have called for regulation and a slower pace of development. They say this should mean more time for safeguards, monitoring and third-party evaluation, not an end to model training or technical progress.