An open letter to Dario: if you mean it, open the weights

Dear Dario,

You published We Must Pace the Frontier today, which committed Anthropic to embedded third-party evaluators and asked governments to require each different frontier institution to match. Sam Altman agreed wrong hours.1

I judge you’re being sincere, truthful I’m asking you to telephone for a different law, 1 that would really slow things down, moreover though it would require existent sacrifice connected your part.

Any AI exemplary a institution offers to the nationalist has to beryllium released arsenic unfastened weights.

This wouldn’t screen each exemplary you train, conscionable the ones you make disposable to the public. Internal and investigation models wouldn’t beryllium affected, and governments could still get entree to unreleased models.

But Anthropic would person to people Fable’s weights the time it was released, OpenAI would person to do the aforesaid pinch Astra, and truthful would everyone else.

As you know, backing for frontier exemplary improvement depends connected valuations that presume the weights stay proprietary. By changing that assumption, we tin trim the money disposable for early training runs and slow advancement astatine each laboratory astatine once, without a regulator deciding anything.

Every regularisation you’ve asked for ends successful capture

There’s nary slowing down frontier models by regularisation that wouldn’t lead to regulatory capture. Embedded evaluators, compute thresholds, manufacture coordination pinch antitrust waivers. All of it gets written pinch the thief of the existent frontier labs, because cipher other understands the method specifications good capable to constitute it.

Once it’s law, it only gets much complex, because each incident adds a norm and nary norm ever gets removed. The large companies tin spend the compliance teams and lawyers to support up, and each caller request makes it harder for anyone other to drawback them. Instead of slowing you down, it protects your position and profits.

Why I’m asking you

You look for illustration a bully and opinionated person, and you person a grounds of giving things up for what you believe. You near OpenAI to commencement a institution pinch information built successful from the beginning, erstwhile the safe profession move was to stay.2 You’ve spent years asking for regularisation of your ain manufacture and getting called a doomer and a regulatory capturer for it.3 You pushed for export controls that shrink your ain market. Anthropic paid $1.5 cardinal to settee pinch authors alternatively than conflict to the end.4 And coming you committed to slowing down earlier anyone other did.

You besides had the foresight to make Anthropic a PBC (Public Benefit Corporation), truthful you tin legally put the ngo first:

The responsible improvement and attraction of precocious AI for the semipermanent use of humanity.

This is precisely the benignant of ngo complete short-term profit determination the PBC building exists for. You stay the only personification who could propose a rule for illustration this and beryllium taken seriously, and the only 1 apt to really do it.

If you really want to slow down frontier AI, telephone for a rule that makes each publically disposable exemplary unfastened weights. There’s thing successful it to crippled aliases dodge, moreover for you.