Jack Clark, co-founder of Anthropic, is sounding the alarm about the need for the AI industry to create ways to slow down or even halt the rapid development of advanced AI. He warns that we're nearing a point where these systems might improve themselves without any human input.
In a recent blog post, he co-wrote with Marina Favaro and in various interviews, Clark pointed out that AI models are quickly moving toward something researchers refer to as recursive self-improvement.
That's when systems could design and produce more advanced versions of themselves. While that kind of tech could lead to major breakthroughs in areas like science and healthcare, it also raises the risk that humans may lose control over increasingly capable AI systems.
He likened the current situation to driving a car that only has an accelerator. "The AI industry has a gas pedal, but it doesn't have a brake pedal," he remarked, suggesting that both governments and tech companies need to collaborate on creating safeguards to help slow development if it becomes necessary.
Anthropic estimates that around 80% of the code behind its AI assistant, Claude, is already generated by AI. Clark hinted that we might see that figure reach close to 100% in just two years.
He emphasized that the industry should start working on regulatory frameworks and technical controls before self-improving AI becomes a reality.
This warning comes as Anthropic gears up for a potential initial public offering, with a valuation nearing $1 trillion. Even as they call for stronger safety measures, the company continues to heavily invest in AI research and development, advocating for broader cooperation on safety standards across the industry.
Clark acknowledged the huge benefits that AI could bring but stressed that we need to have a serious discussion about its economic and social ramifications, like potential job losses and the long-term challenge of keeping advanced AI aligned with human values.
For comments and feedback: editorial@rttnews.com