Anthropic has called for major AI labs to consider a coordinated and verifiable pause in AI development, noting that advancements could soon lead to AI systems that self-improve beyond society’s ability to manage risks. The company’s Claude creator highlighted that AI’s capacity to perform tasks independently is doubling approximately every four months, indicating a potential shift toward “recursive self-improvement,” where AI could enhance itself without human help. Anthropic emphasized the importance of managing the safety and behavior of such systems and argued that a pause would give society time to address the implications of these changes.
The fear that advanced AI could become uncontrollable and harmful has grown as technology advances. Anthropic pointed to its own Mythos model, which raised concerns in various sectors for its ability to identify weaknesses in existing code. Despite growing concerns, regulatory action has been slow, particularly in the U. S., where an executive order recently placed responsibility on labs to voluntarily submit powerful models for government testing before public release.
While AI researchers have previously called for pauses, there has been limited success. Notably, Elon Musk supported a 2023 initiative to halt AI development for six months to establish safety measures. Anthropic has positioned itself as a safety-oriented lab and resisted military use of its models for surveillance and autonomous weapons, which led to backlash from the government.
Anthropic warned that a unilateral pause by one lab would not be effective, as it could lead to new front-runners while failing to create a necessary wider deliberative process. The company plans to engage various stakeholders, including policymakers and rival firms, to discuss the risks associated with AI self-improvement and explore supportive systems for a slowdown.
With information from Reuters
