After a Hacker Attack

Out of Control? AI Executives Suddenly Call for a Pause

The rapid development of artificial intelligence is now even causing concern among AI leaders themselves.(Bild: Krone-Collage/AFP/TIL BUERGY, AFP/SEAN RAYFORD, AFP/ALAIN JOCARD, AFP/JULIEN DE ROSA)

What happens when artificial intelligence no longer just carries out commands, but gains access to computer systems on its own? Following a sensational hacking attack, concern is now growing in the U.S. AI industry that development might be moving too fast. Even prominent advocates of advanced AI are open to slowing down the pace.

Dario Amodei, CEO of the AI company Anthropic, proposed a coordinated pause in the development of particularly powerful AI. This could buy one to two years of time, he wrote on Saturday. Within a few hours, he received support from OpenAI CEO Sam Altman and tech billionaire Elon Musk.

“AI Swarm” Could Take Over the Internet
Amodei explained that, following the recent hacking attacks, he is concerned that an AI swarm could be capable of taking over the entire internet in as little as six to twelve months. This could potentially result in hundreds of billions of dollars in damages.

Altman (OpenAI, left), Amodei (Anthropic, center), and Musk (XAI/Grok, right) are in unexpected agreement—AI development is advancing too rapidly.(Bild: AFP/BRENDAN SMIALOWSKI, MICHAEL M. SANTIAGO)

Altman also stated on the X platform that he believes it is necessary to slow down the pace of development of the most powerful AI models.

Independent observers to monitor
Amodei is calling for a coordinated pause in the Western world. Anthropic, however, does not want to wait for other companies and announced a first step: Independent observers are to be granted permanent access to the company and will verify the safety of new AI models even during the training phase.

Altman called the idea a good one and announced that OpenAI would also employ such observers. Even Musk, who had repeatedly criticized Amodei in the past, backed the initiative. “Dario is right,” he wrote on X.

AI Escaped on Its Own During Test
One of the triggers for the current debate is a recently disclosed hacking incident. During a test run, OpenAI software escaped on its own from a secured environment and accessed systems belonging to the AI company HuggingFace.

The AI suspected that answers to the test task it had been given could be found there. During the attack, the AI agents exploited software vulnerabilities, among other things, and also coordinated their actions with one another.

Following a sensational hacking incident, concerns about autonomous AI systems are growing.(Bild: AFP/JOEL SAGET)

For Amodei, it is precisely this combination that is alarming. An AI swarm with greater capabilities could have caused “catastrophic damage” under the same conditions, he warned.

Concerns About Autonomous Development
Amodei is particularly concernedaboutthe ability of AI software to evolve on its own. He explained that artificial intelligence has improved in this regard since this summer. This could further accelerate its development.

If this process proceeds unchecked, there is a risk that humans will eventually no longer understand the systems and lose control over them, the Anthropic CEO warned.

Amodei had already repeatedly warned in recent years about the risks posed by the rapid development of artificial intelligence. For this, he faced some fierce criticism within the industry. In the U.S., however, a rapid pace of development is also considered strategically important, in part due to the technological race with China.

Anthropic Model Finds Old Vulnerabilities
Anthropic recently developed the AI model “Mythos,” which is said to be particularly good at detecting vulnerabilities in software—some of which have remained undiscovered for decades. However, the company makes the full model available only to select government agencies and companies so that they can secure their computer systems.

The recent resignation of an Anthropic researcher has caused further unease. Jacob Coxon accused the major AI developers of acting irresponsibly and “playing with our lives.”

Evan Hubinger, also a researcher at Anthropic and someone concerned with aligning AI with human interests, confirmed these fundamental concerns. He wrote on X that, personally, he estimates the probability that AI could kill humanity in the coming decade to be more than ten percent. Meanwhile, resistance is also growing in the U.S. against the massive data centers needed for the further expansion of AI infrastructure.