Why it matters. One of the world's leading AI developers is publicly arguing the entire industry, including itself, is moving too fast to stay in control — a rare call for restraint from inside the race.
Background. Anthropic is a US AI firm founded by former OpenAI researchers and known for marketing 'safety' as a differentiator; it recently surpassed OpenAI in valuation and started moving toward an IPO. This story was widely picked up in South Korea, where major papers like Hankyoreh (progressive) and Kyunghyang Shinmun (progressive) closely track global AI-governance debates that bear on Korea's own tech sector.
What to watch next. Watch whether Anthropic's promised coalition of policymakers and civil society gains any traction with rival labs, who have shown little appetite for a coordinated pause.
Anthropic, the US artificial-intelligence company behind the Claude chatbot, on June 4 publicly urged AI labs around the world to consider deliberately slowing the pace of development, warning that systems are advancing so quickly they could soon begin improving themselves without human oversight. The call came in a blog post co-written by Anthropic co-founder Jack Clark and Marina Favaro, who leads the company’s in-house research arm, the Anthropic Institute.
The fear: AI that improves itself
At the center of the warning is a concept the authors call “recursive self-improvement” — the point at which an AI system becomes capable enough to autonomously design, build and train its own successor models. In that scenario, the company wrote, a new version of Claude could be created from an earlier one with no human in the loop.
Anthropic stressed that the industry has not reached this stage, and that it is not inevitable. But it cautioned that the milestone “could arrive earlier than most institutions have prepared for.” Such a leap, the post argued, could deliver “enormous benefits to science and medicine” while simultaneously raising the risk that humans lose control over the AI systems they build.
To buy society time to adapt, the authors proposed slowing or temporarily pausing frontier AI development so that social structures and safety research can catch up. Specifically, they suggested that labs across multiple countries agree to halt work under “the same conditions,” with each able to verify that the others are genuinely complying — a mutual-inspection model rather than a unilateral pause.
A Cold War precedent — and its limits
To show such coordination is not impossible, the post pointed to an arms-control precedent: the 1987 Intermediate-Range Nuclear Forces (INF) Treaty between the United States and the Soviet Union, which banned an entire class of ground-launched ballistic and cruise missiles with ranges of 500 to 5,000 kilometers and became a symbol of the Cold War’s end. Anthropic noted a sobering caveat: “These regimes took decades to build the infrastructure and trust, but we don’t have that long.” The company also acknowledged the treaty’s own fate — it effectively collapsed after the US withdrew in 2019 and Russia followed.
Anthropic says it plans to convene a body, working with policymakers and civil society, to discuss how such coordination on recursive self-improvement might actually be arranged.
“An accelerator but no brakes”
In a separate interview with Britain’s BBC, Clark put the concern more bluntly, saying the AI industry today is “like having only an accelerator pedal and no brakes.” There should, he argued, be an option “to take your foot off the accelerator and apply the brake” — in other words, to slow development enough to plan for AI’s impact on existing industries and on AI itself.
The two Korean outlets covering the announcement framed it slightly differently. Hankyoreh foregrounded the governance and treaty dimension, emphasizing the verification mechanism and the historical parallel. Kyunghyang Shinmun leaned into the business backdrop, noting that Anthropic has consistently positioned itself around AI “safety” even as it competes commercially.
That context is notable. Anthropic recently recorded a higher corporate valuation than its rival OpenAI and has begun the process toward an initial public offering (IPO). It has released its high-performance model, Mythos, only in limited fashion, citing cybersecurity risks such as AI-enabled hacking. The company has also argued that its Claude large language model should not be used for domestic surveillance or fully autonomous lethal weapons — a stance that has put it at odds with the US Department of Defense.
The appeal lands at a moment when the people warning loudest about AI’s dangers are, in many cases, the same companies racing to build it. Anthropic’s pitch is that a coordinated, verifiable slowdown — not a halt by any single lab — is the only realistic way to keep humans in control.
