Anthropic co-founder calls for AI brake pedal as systems near self-development

Anthropic co-founder calls for AI "brake pedal" as systems near self-sufficiency

Jack Clark, co-founder of AI safety company Anthropic, has warned that artificial intelligence is approaching a point where it can develop without human oversight. He called for regulatory frameworks to slow AI progression before that threshold is reached.

"Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal," Clark told BBC Newsnight. He stressed that governments need to establish policies that keep humans in control of increasingly powerful AI systems.

Self-improving systems within reach

Anthropic's Claude chatbot currently writes 80% of its own code. Clark said reaching 100% is possible within two years, which he described as having "huge implications" for how AI systems develop.

The concern reflects a technical reality: as AI systems become more capable, they require less human direction to improve themselves. This creates a feedback loop where human involvement diminishes with each generation.

Regulatory model, not specifics

Clark drew a parallel to how governments regulated the oil industry a century ago. "Society's response was to come up with a sensible policy and regulatory framework," he said, adding that similar structures are needed for AI.

He did not specify how a brake mechanism would work in practice. His framing suggests the solution lies in government policy rather than technical safeguards built into the systems themselves.

Gap between rhetoric and action

Anthropic welcomed President Trump's recent executive order on AI, which took a hands-off approach and did not require mandatory government safety testing. Major AI companies, including Anthropic, OpenAI, and Google, have not announced plans to pause their own research.

Clark said Anthropic's public warnings about AI risks stem from a desire to inform society about what the company observes internally. "I am worried for my kids if we as a society don't have a serious conversation about what the implications of AI's continued advances mean," he said.

Economic disruption and workforce change

Clark identified job displacement as a near-term risk. Tech companies have conducted mass layoffs over the past year, often citing AI's ability to perform work previously done by hundreds or thousands of engineers.

However, he suggested that human creativity may remain difficult for AI to replicate. "There are open questions about whether AI systems can be truly creative," Clark said. "There is not really evidence for that yet."

He advised workers concerned about AI's economic impact to develop creative skills and pursue broad education. "People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this," he said.

Company context

Anthropic, founded in 2021 by Clark, Dario Amodei, and others, is preparing for a public stock listing. The company is valued by private investors at nearly $1 trillion, positioning it as one of the most valuable IPOs in history and among the first major generative AI and LLM companies to go public.

The company has positioned itself as vocal about AI risks, including a public dispute with the U.S. Department of Defense over concerns that its tools could enable mass surveillance and autonomous warfare.