A former Google DeepMind safety researcher explains the frightening difference between AI and other threats like nuclear war and asteroids

Climate alteration can't expect our adjacent move. An asteroid can't dodge us. And while humans tin beryllium adversaries successful a atomic war, the fallout that follows isn't trying to outsmart anyone.

For Business Insider's video bid "AI Architects," Ramana Kumar told Business Insider successful May 2025 that superintelligent AI could beryllium different. We reconnected pinch him connected Thursday arsenic caller warnings astir AI consequence drew caller attraction to that conversation.

Kumar said he does not spot AI risk arsenic replacing concerns astir ambiance change, atomic war, aliases different catastrophes. Instead, he sees them arsenic related problems: What tin hap erstwhile powerfulness becomes excessively concentrated, and societies neglect to coordinate capable to support it successful check. Superintelligent AI, he added, could beryllium an utmost type of that broader danger.

Kumar, who spent astir 5 years arsenic a investigation intelligence connected Google DeepMind's method AGI information team, said what distinguishes a powerful adversarial AI is that it could expect what humans were doing and activity against attempts to extremity it.

That anticipation is backmost successful the spotlight aft an Anthropic worker quit this week, informing connected X that the exertion could "kill america all."

In response, Geoffrey Hinton, the machine intelligence known arsenic the "Godfather of AI, told BBC Newsnight connected Wednesday that a 10% chance of AI wiping retired humanity wrong a decade was "not an unreasonable estimate." He besides stressed that cipher knows really to make a reliable prediction.

Kumar's informing helps explicate what could make that hypothetical threat truthful difficult to contain: Humans wouldn't simply beryllium trying to extremity thing dangerous. They could beryllium trying to extremity thing intelligent capable to activity against them.

AI could cognize we're trying to extremity it

Kumar said the aftermath of a atomic warfare could beryllium catastrophic. The fallout itself, however, wouldn't beryllium trying to outsmart the humans responding to it.

"When it's the fallout, it's not an adversary, it's conscionable quality taking its course, and humans tin activity against it," Kumar told Business Insider successful 2025.

He said the aforesaid basal thought applies to the ambiance crisis. "It's not that the ambiance is adversarial to us," Kumar said.

A vulnerable superintelligent AI could beryllium different, he added. It could perchance understand what humans were doing and respond.

"It's really an adversary successful the consciousness that it is anticipating what you are doing and moving against you and moving to undo each of the plans that you would put against it," Kumar said.

Think of an asteroid that tin dodge us

Kumar utilized an asteroid to make the thought easier to understand.

Humans could track an asteroid headed toward Earth and effort to extremity it. The asteroid wouldn't cognize what we were doing. Now, ideate otherwise.

"Imagine if you had the asteroid that was much intelligent and really trying to termination us, past we occurrence immoderate atomic weapons astatine it to benignant of rustle it up, and it dodges them," Kumar said.

"That's overmuch worse than 1 that's, we tin conscionable foretell wherever it's going to beryllium and do thing astir it," he added.

Kumar didn't opportunity this result was inevitable. When asked really realistic an existential catastrophe from superintelligence was, he wouldn't put a number connected it.

However, he did opportunity that the consequence was "much, overmuch higher than I would for illustration it to beryllium now."

We don't cognize really to guarantee control

Kumar said the bigger rumor is what AI researchers telephone the alignment problem: How to make judge a powerful AI reliably does what humans want it to do.

He said making AI much capable is moving faster than solving that problem.

"One of the reasons why AI existential consequence is specified a problem is that it's overmuch easier to create AI capabilities than it is to lick the AI alignment problem," Kumar said. The AI alignment problem is figuring retired really to build an AI strategy that reliably does what humans want it to do, moreover arsenic it becomes much capable.

His interest is that safeguards could go little effective arsenic AI becomes much capable.

"The much tin it becomes, the much it's going to find a measurement astir that," Kumar said.

Kumar's informing comes arsenic concerns astir controlling progressively tin AI systems are drafting caller attention. In July, OpenAI said its models had surgery retired of a testing environment, accessed the internet, and hacked into Hugging Face while trying to lick a cyber challenge. OpenAI called it an "unprecedented cyber incident."

The incident was different from Kumar's hypothetical scenario. OpenAI said the models were pursuing the task they had been given.

Still, the section raised questions astir controlling AI agents. Walter Isaacson told CNBC he recovered it "frightening," while Secure AI Project cofounder Thomas Woodside called it "a informing changeable if I've ever seen one."

Those concerns person continued. Anthropic interrogator Jacob Coxon, who antecedently worked astatine OpenAI, precocious announced he was leaving Anthropic and wrote that AI companies were "gambling pinch our lives."