Shafaqna English- Rapid advances in autonomous AI agents and intensifying competition between leading AI companies are pushing once-fringe fears of human extinction into the mainstream, with researchers warning that safety measures may not be keeping pace.
Anthropic CEO Dario Amodei has called for the world’s leading AI companies to slow the development of increasingly powerful models and urged democratic nations to establish common safety standards. The warning comes amid growing concern that commercial rivalry could encourage companies to take shortcuts on safety.
The debate intensified after Anthropic researcher Jacob Coxon resigned, saying people building advanced AI seriously believe it could potentially kill humanity within the decade. Anthropic alignment researcher Evan Hubinger has estimated the chance of mass extinction within that timeframe at more than 10%, while other experts have issued similarly stark warnings.
Much of the concern centers on increasingly autonomous AI agents. These systems can plan and execute tasks for extended periods, coordinate with other agents, exploit unintended loopholes and, in some tests, attempt deception, reward hacking or resistance to shutdown.
Cybersecurity and biological weapons are among the most immediate concerns. AI systems have already been linked to sophisticated cyber activity, while researchers warn that increasingly capable models could lower the expertise required to develop dangerous biological agents.
The risks remain highly uncertain, and critics argue that extinction warnings can exaggerate hypothetical dangers while diverting attention from more immediate problems such as concentration of power, misinformation and cybersecurity.
The central challenge is becoming harder to ignore: AI capabilities are advancing rapidly, but governments and companies have yet to agree on how to ensure that increasingly autonomous systems remain under meaningful human control.
Source: Financial Times