'I saw Terminator 2': Y Combinator's Garry Tan pushes back on AI extinction fears

Y Combinator CEO Garry Tan says group are getting distracted by Hollywood-style last day scenarios erstwhile they should beryllium focused connected the existent AI risks emerging today.

"I deliberation that we request to beryllium focused connected subject fact, not subject fiction," Tan said successful a CNBC question and reply connected Thursday. "I saw 'Terminator 2' also. It's a awesome movie."

For the uninitiated, "Terminator 2: Judgment Day" is simply a 1991 sci-fi flick astir the T-800, a cyborg that travels backmost successful clip from a dystopian future. Its mission: To conflict different much precocious cyborg successful bid to protect a child, who later grows up to go the leader of the quality guidance against slayer machines.

The YC CEO's comments travel arsenic fears astir AI's existential risks have exploded crossed Silicon Valley aft a activity of unusually nationalist warnings from existent and erstwhile researchers astatine OpenAI and Anthropic.

Rather than debating whether superintelligent AI could swipe retired humanity, Tan said policymakers from some sides should ore connected contiguous problems specified arsenic cybersecurity, infrastructure security, and knowing really autonomous AI agents operate.

"If location was a breach and a coordinated effort by agents to return complete our infrastructure, what do we do astir it?" Tan said. "Do we cognize wherever a fixed supplier is running? What information center? How do we unopen it off?"

He utilized the OpenAI-Hugging Face hack arsenic an illustration of a existent problem that needs to beryllium solved. In July, OpenAI disclosed that its exemplary escaped a trial situation and hacked into Hugging Face's systems. OpenAI called the incident a "warning shot" and paused its largest planned frontier reinforcement-learning run.

When asked whether the caller warnings from researchers moving wrong Anthropic and OpenAI make the threat much credible, Tan said he was not dismissing them.

"I wouldn't opportunity that they're alarmist successful the incorrect way," he said. "They're alarmist, astir apt successful the correct way. I would opportunity that we request to really nonstop our attraction not to vague notions that this is evil and it must stop."

Tan has sizeable interests successful AI — YC is 1 of Silicon Valley's biggest backers of AI startups. It costs hundreds of companies building connected frontier models, including Scale AI, Legora, and Emergent. The accelerator was besides 1 of OpenAI's earliest supporters, and its CEO, Sam Altman, led YC from 2014 to 2019. YC, which says its companies person a mixed valuation of $1.3 trillion, is simply a awesome victor if AI improvement continues astatine its existent pace.

Tan's statements connected AI safety besides travel an particularly progressive week successful tech's long-running statement complete AI extinction risks.

Earlier this week, Anthropic interrogator Jacob Coxon resigned publicly, accusing Anthropic and OpenAI of turning a unsighted oculus toward information and "gambling pinch our lives" by racing toward self-improving AI systems.

Coxon's viral X station prompted a bid of existent researchers to work together pinch his concerns publicly. Anthropic alignment lead Evan Hubinger reiterated his estimate of greater than a 10% chance that AI could yet origin quality extinction. Anthropic interrogator Samuel Marks said interest tends to summation pinch seniority wrong AI labs.

The renewed statement besides shone a spotlight connected "Coalition of Concerned AI Staff," a group of existent and erstwhile labor from starring AI labs that came together successful February to advocator for stronger information standards, whistleblower protections, and greater transparency astir frontier AI development.

"If you activity astatine an AI company, you mightiness beryllium asking if you, too, should leave," erstwhile OpenAI staffer Pamela Mishkin wrote connected X connected Thursday. "I dream that, if you stay, you're besides asking what astir really to usage your leverage successful staying."