Gemini 4 Neon looks like an Opus 5.5 competitor

Hacker News by 2 min read 506x views
Gemini 4 Neon looks like an Opus 5.5 competitor

Share Post

TLDR: we project Gemini 4 Neon’s achievement according to what a faster Gemini 4 Carbon power mark and discover it equal to Opus 5.5

Gemini 4 Carbon is the finest example on the Agent Duel leaderboard if you exclude first voting. Carbon’s chief issue is that it is slooooow.

Today, Gemini 4 Neon was released, and looks to be an incremental benchmark betterment alongside a extremely ample speed improvement. Earlier today, it was being served extremely accelerated via OpenRouter: 127 tokens per second. This is way faster than Carbon at 46 tokens per second.

This got me wondering, could Neon be not fair a fine example but the finest model?

Modeling our leaderboard

To investigate, I modeled the win charge in our leaderboard. The finest example has three components, example identity, speed and whether the person voted early. My first modeling did poorly, so I added an communication between speed and first voting. This ended up operating well.

Win charge vs. Projected win rate

And surprisingly, speed seems to be beautiful much as crucial as example character (another blog on that soon).

Feature importance

Projecting Gemini 4 Neon performance

If you assume the identical achievement as Gemini 4 Carbon (which is sensible stated equal improved benchmark scores), and 127 tps, we would project an Elo on the Agent Duel leaderboard of 1,066, fair below Opus 5.5 at 1,078 in our afterward inspection snapshot.

Synthetic Neon Elo projection

The 90% duration for the projected Elo difference between Gemini 4 Neon and Opus 5.5 is -94 to +68, alongside Neon projected to win 48.3% of the period against Opus (counting ties as fractional a win).

Other Article Hacker News
↑
Close Right Ads
Close Left Ads