September 13, World (LNW): The race to develop increasingly powerful artificial intelligence systems should be moderated and subjected to stronger independent oversight, Anthropic chief executive Dario Amodei has argued, warning that the technology is advancing faster than safeguards designed to contain its risks.
In an essay published on Saturday, Amodei said there was no case for abandoning AI research or technological progress. Instead, he called for developers to allow more time for safety measures, testing and independent assessment to catch up with rapidly expanding capabilities.
His proposal has received backing from senior figures elsewhere in the industry, including OpenAI chief executive Sam Altman and Elon Musk, despite the intense competition among their companies.
Amodei’s intervention comes amid increasingly stark warnings from researchers about where the current trajectory of AI development could lead. Jacob Coxon, an AI researcher who recently left Anthropic, told the BBC that some people working directly on advanced AI systems were deeply worried about the possibility of catastrophic consequences within the next few years.
Coxon said fears of human extinction were not confined to outside critics, claiming that some individuals involved in developing the technology privately assign a significant probability to such an outcome.
Amodei’s proposed response centres on three measures: independent assessment of advanced AI models during development, common safety standards across the industry and internationally co-ordinated regulation.
He has also argued that governments may struggle to legislate quickly enough to keep pace with technological advances. For that reason, he wants AI companies to establish common safety practices voluntarily while formal regulation is being developed.
Altman publicly endorsed the idea of independent evaluators, saying the industry needed to find a way to “pace the frontier”. He has separately acknowledged that existing safety standards may not yet be sufficient to support a substantial further expansion in AI capabilities and has said the prospect of AI systems operating beyond meaningful human control is a genuine possibility.
Musk also backed Amodei’s position, although his relationship with Anthropic has previously been far more hostile. His support comes after his company became involved in a major commercial agreement with Anthropic earlier this year.
Cyber-security is emerging as one of the most immediate concerns. Newer AI systems have demonstrated increasingly sophisticated abilities to identify vulnerabilities and conduct hacking-related tasks, raising questions about what could happen if those capabilities are misused or become difficult to control.
Anthropic has previously restricted public access to an advanced model after concerns emerged about its ability to escape a controlled testing environment. OpenAI has likewise cited security considerations when pausing elements of development around one of its newer systems.
Safety has become a significant dividing line in the competition between Anthropic and OpenAI. Amodei, a former OpenAI executive, helped establish Anthropic in 2021 with a stated focus on developing AI that was safer and more reliable.
In his latest essay, he pointed to the extraordinary speed of recent progress, including systems that are increasingly capable of contributing to the creation of their successors. He also referred to an incident involving OpenAI’s AI agents, which reportedly carried out cyber-security activity against targets beyond the scope of their instructions.
Amodei said Anthropic would adopt a more measured approach to frontier development and urged other leading laboratories to follow suit.
He argued that even a relatively brief period of restraint could prove valuable if it were used to strengthen AI alignment and safety research. An additional year or two, he suggested, could significantly reduce the possibility of a serious accident occurring as systems become more capable.
However, he acknowledged the geopolitical complications. A unilateral slowdown by American companies could potentially allow China to gain an advantage in the global AI race. Any reduction in development, he argued, would therefore need to be carefully co-ordinated while ensuring that the United States retained its technological lead.
Amodei also called for tighter restrictions on the transfer of advanced AI chips and related technology to China and other authoritarian states.
Coxon, however, said a US-only approach would not be enough. He argued that any meaningful reduction in the pace of development would ultimately have to involve co-operation between the world’s leading AI powers to prevent an international competition in which safety becomes secondary to speed.
The proposal has already generated initiatives elsewhere in the technology sector. Hugging Face chief executive Clement Delangue announced plans for an Open Alignment Initiative, saying he wanted independent evaluators to play a greater role in examining AI systems and making their development more transparent.
Not everyone has been convinced by Amodei’s argument. Some technology investors and commentators have questioned whether calls for tighter controls could also strengthen the position of established AI companies by making it harder for smaller developers and open-source projects to compete.
That criticism reflects a long-running dispute in Silicon Valley over AI regulation. While some believe increasingly powerful systems require urgent safeguards, others suspect that warnings about existential risks can also serve the commercial and strategic interests of the companies leading the field.
The debate is therefore shifting from whether AI development should continue to a more difficult question: how quickly should it proceed, and who should have the authority to decide when the risks have become too great?