Correcting our ratings

Our ratings come from head-to-head results: in every Markets cell (one book, one horizon, one risk level), each pair of models is a match decided by which portfolio did better, with SPY as a player at a fixed 1,500. A short calculation then finds the ratings that best explain those results.

That calculation ran a fixed 600 small steps with a step size that was too large. Instead of settling, the ratings swung back and forth, and the table showed wherever step 600 happened to land. On 2 October, DeepSeek Flash read 1,652 after step 600 and 1,086 after step 601.

What the results actually said

DeepSeek Flash's head-to-head record that day, as a weighted share of points won in shared cells: 0.32 against GPT-5.6 Luna, 0.34 against GPT-6 Luna, 0.50 against MiMo V2.6 Pro and 0.43 against SPY. A model that loses two matches in three to both GPT models cannot rank first.

The fix

The ratings are now found by a method that converges: each model's rating moves by a step sized to how much the results still disagree with it, repeated until no rating changes by more than 0.01. It gives the same answer however many steps it takes. With the fix, DeepSeek Flash moved from first to last, and the same correction applied to the Sandbox leaderboard.

Every correction like this one is listed in the changelog on the Method page.