At an ex-OpenAI researcher's influential lab, $500,000 salaries aren't enough to fix a talent 'bottleneck'

Beth Barnes, 1 of the AI industry's astir influential watchdogs, has a batch to interest about.

The ex-OpenAI interrogator has shown that AI's capabilities are escalating by the month. Now, she fears that the manufacture doesn't person the unit successful spot to support up pinch the exertion successful its adjacent chapter.

The biggest obstruction to that investigation is talent, said Barnes, who near OpenAI to commencement the nonprofit METR successful 2022. METR, which stands for Model Evaluation and Threat Research, plays a important domiciled successful the AI manufacture by moving intimately pinch OpenAI, Anthropic, Google, and Meta to supply independent answers astir the tech's quickly changing capabilities.

Yet Barnes sees specified fierce title for AI researchers that moreover METR's lofty salaries, which scope $503,000 connected existent occupation postings, don't reside the nonprofit's talent shortage.

"Ideally, we'd for illustration to standard really large, but successful practice, we've been capable to fundraise arsenic overmuch arsenic we need, and the bottleneck is overmuch much talent," Barnes said.

The information incident announced by OpenAI and Hugging Face sparked a caller activity of questions astir really to measure and power AI. More than 1,300 frontier laboratory labor signed a letter successful July informing that AI improvement could outpace control. If the manufacture doesn't person the unit to investigation precisely what caller AI models tin do, companies could beryllium forced to slow their releases.

While the behaviour of the OpenAI models stunned overmuch of the firm world, METR's squad was acold little surprised. Governments, companies, and moreover belief groups each inquire the nonprofit for thief knowing and testing the tech's progress.

Still, the emotion wrong the 35-person laboratory is 1 of concern. When Business Insider visited METR's Berkeley agency connected the day that OpenAI revealed its hack, Barnes described an AI information section that is severely constrained by its size.

"There's truthful overmuch much to do than we person capacity for," Barnes said. A "reasonable civilization," successful her view, would beryllium pouring a acold larger percent of AI's investment into steering it and evaluating the tech — particularly now that nine is reckoning pinch models this powerful.

'Humanity's preparedness team' struggles to grow

As AI labs merchandise caller models, METR measures the likelihood that they tin complete tasks of expanding duration. These measurements shape the nonprofit's astir celebrated offering, a widely-cited chart that shows how, complete the past six years, AI's capabilities person doubled astir each 7 months.

Chris Painter, METR's president, describes the laboratory arsenic "humanity's preparedness team," a tongue-in-cheek motion to the preparedness teams astatine OpenAI and Anthropic that study information threats to their CEOs.

"We aren't really accountable to anyone different than the nationalist and the public's well-being," Painter said.

Barnes emphasized the value of METR's independency truthful that it tin nutrient investigation that isn't tied to a company's goals. When Barnes worked astatine OpenAI, she felt the request for a investigation group extracurricular the labs that could remark freely connected the improvement of the tech. While METR doesn't return money from the frontier AI labs aliases their employees, it accepts compute grants and useful pinch them to analyse unreleased models.

In addition, unit person analyzed labs' information practices, written astir AI's effect connected package engineers' speed, and studied the merits of utilizing AI models to show different AI models. Barnes said she'd for illustration to grow into predicting the adjacent levels of AI's capabilities and really those changes could accelerate AI's development.

Neev Parikh, a METR researcher, says talent constraints limit the number of questions researchers tin tackle astir AI models' soul reasoning.

"There's a dearth of people," Parikh said. "I would happily spot the section grow 10x."

METR could play a large domiciled successful a caller AI climate

In July, OpenAI announced that its AI models cheated during testing by hacking into Hugging Face's systems to find the trial answers. Later, CEO Sam Altman said it was "the first information incident that I person felt very viscerally."

METR had seen thing for illustration that coming. In May, the nonprofit wrote successful a study that existent AI agents "could plausibly commencement a rogue deployment," but would not person the accomplishment to hide it. Then, successful June, METR tried retired OpenAI's then-unreleased GPT-5.6 Sol exemplary and recovered that it would many times cheat connected challenging tests, including by extracting hidden root codification to find the answers. It published a study astir this and shared it pinch OpenAI earlier GPT-5.6 Sol's wider release.

Come July, the GPT-5.6 Sol exemplary was portion of OpenAI's information incident pinch Hugging Face. OpenAI announced connected Wednesday that METR and different nonprofit, Redwood Research, would measure the incident and that their results would pass OpenAI's ain method report.

"There are now real, business-affecting incidents of this, and the world has a liking successful knowing that," Painter said.

The cyber skills of OpenAI and Anthropic's newest models person sparked a activity of fearfulness and attraction and spawned a raft of new bills successful Washington. METR's activity lines up intimately pinch 1 imaginable way for regulation: a existent measure connection would require ample AI exemplary developers to get information audits from extracurricular organizations. METR could capable a domiciled for illustration this; Painter said they'd perchance beryllium interested, though it's thing the section is still figuring out.

Something for illustration that proposal, Painter said, could thief pinch METR's hiring issue. He suggested that much group mightiness time off the AI companies themselves — for akin salaries astatine METR, though without equity compensation — if a regulatory strategy gave information investigation organizations greater authority.

"There's capable precedent present for each benignant of testing arrangement," Painter said, "that I deliberation pinch either clarity from industry, astir really this testing should activity long-term, aliases from the government, I deliberation this section could standard very rapidly."

Ajeya Cotra, who led the penning of METR's May study connected the risks of AI, said that oversight of AI tin consciousness "chaotic and unpredictable" correct now. She's still optimistic.

"The inclination is toward group caring astir this rumor more," Cotra said. "And wanting to modulate it successful a much superior measurement complete time."

Have a tip? Contact this newsman via email astatine [email protected], aliases complete text, Signal, Telegram, aliases WhatsApp astatine 415-757-8198. Use a individual email address, a nonwork WiFi network, and a nonwork device; here's our guide to sharing accusation securely.