Rather than investing in a brute-force RL climb, we carefully iterated on our SFT and RL recipes alongside a concentration on compute efficiency. We opted not to buy any posttraining data, alternatively devising our own synthetic data pipelines, norm filtering, and safety metrics.
Our blended RL formula uses
,
, and
alongside high-quality long-horizon synthetic environments. The RL tasks are tuned for our example capabilities, and we continuously added harder tasks as the example improved.
The end outcome was a steep RL ascend from a powerful first SFT checkpoint. We reached a TerminalBench 2.1 mark of 80.5 alongside lone 1.4M rollouts, approximately 70x small rollouts than Reflection Beam, the previous SOTA.
.png)
