(Bloomberg) — Anthropic PBC called for the creation of an industrywide mechanism to pause the development of artificial intelligence, giving society the opportunity to “deal with its immense implications.”
Most Read from Bloomberg
The technology is advancing to the point where AI will make human labor thousands of times more efficient or even replace it, creating a new set of risks, the company said in a lengthy blog post published Thursday by co-founder Jack Clark and Anthropic Institute lead Marina Favaro.
AI could begin to improve itself and build its own successors, diminishing the role of humans, Clark and Favaro wrote. “That collision, where recursive intelligence building itself ever faster meets the world of humans, relationships, and governance, is another part of this future we can’t predict,” they wrote.
The blog post compared the idea to the international regulation of nuclear weapons. Any pause mechanism would also need to ensure less scrupulous AI labs don’t secretly keep working to advance the technology during a break, likely through checks by peer labs.
Anthropic plans to organize conversations with policymakers, researchers and other AI companies to discuss a structure for “coordination and deliberation” and will publish the results, according to the post.
The post isn’t the first time a frontier AI researcher has called for a break in development. In 2023, the Future of Life Institute, a nonprofit focused on mitigating risks from technology, called for a pause of at least six months to put guardrails on AI, warning about potentially catastrophic effects. Billionaire Elon Musk signed it along with more than 1,000 other researchers and executives.
Anthropic itself has continued to put out advanced models and tools such as its popular assistant Claude, and the new Mythos model that it says can detect and exploit cybersecurity flaws at remarkable speed. The company is also preparing for an initial public offering.
Critics at the time of the 2023 letter said that stopping development would unnecessarily crimp innovation, and that US companies agreeing to tap the brakes would hand an advantage to other countries. Former Google Chief Executive Officer Eric Schmidt warned that the pause would give an advantage to competitors in China, which is locked in a technological trade war with the US.
