A report co-authored by Nobel laureate Geoffrey Hinton and Yoshua Bengio, two of the “godfathers” of modern AI, published on September 28, warns governments to prepare for what the authors call an AI “intelligence explosion,” according to a report by The Guardian.

The authors are concerned that AI systems could soon begin improving themselves without human intervention, accelerating development faster than governments can respond.

Contributors to the report, titled “What if automating AI R&D triggers an intelligence explosion?“, include more than 20 people, including Anthropic co-founder Jack Clark and OpenAI chief scientist Jakub Pachocki. 

What the Report Warns

The authors urge governments to act now, before acceleration begins, warning that “once an intelligence explosion begins, the window for action may close.” 

The report states that the core concern is “recursive self-improvement,” which is AI systems improving themselves without human intervention. The authors call it automated AI research and development (R&D).

The authors argue that automating AI R&D is the most likely trigger for an intelligence explosion, since AI systems are already helping to improve their own underlying technology. 

Once AI reaches expert-level capability at this, a single developer could effectively run a workforce equivalent to millions of top researchers, the paper says.

Their central recommendation is for governments to gain real visibility into AI R&D automation as it happens, through measures such as transparent progress reports and independent auditors embedded within companies, alongside ways to constrain the pace of development and preparation to adapt if an intelligence explosion occurs.

“An intelligence explosion could be the most consequential technological development in human history,” the paper states, warning it could compress years of progress into months and erode checks on power within and between states, companies, and governments.

The report warns of three main risks:

1. AI-enabled biological and cyber threats could outpace the defenses meant to counter them
2. Reduced human involvement in AI R&D could mean reduced human oversight 
3. States could turn a modest AI lead into a decisive one. 

However, the authors also call these impacts uncertain, noting AI could also help accelerate how fast risks get mitigated.

What This Means for Enterprise AI Oversight

This report is a clear warning from the experts and the people building the technology. 

In this analysis, if AI is approaching the point where it can meaningfully accelerate its own development, AI governance frameworks built around a human reviewing each major step may not hold much longer. 

The report’s core recommendation, real-time visibility into AI R&D paired with independent auditors, suggests AI governance needs to operate at the pace of AI development rather than an annual review cycle. 

An organization whose AI governance still assumes slow, predictable change should treat this as a signal to build in more frequent review points now.