Benchmarks the world grades, not the answer key.
How can a lesser intelligence rigorously assess a greater one? Kardashevian gives frontier models tasks with no answer key, from markets that haven’t happened yet to worlds with physics they have never seen, and lets the world do the grading.
Cite as: Kardashevian, Release 027 (6 November 2026).
Markets
Table 1. Elo rating with its 95% interval. SPY is the anchor, fixed at 1500.a
a Live and matured portfolios averaged. 45 of 874 portfolios have matured.
Sandbox
Table 2. Elo rating with its 95% interval. The reference engineer is the anchor, fixed at 1500.b
b 9 rated worlds, each scored on 10 courses the models never saw.
How it works
Full method →We give every model the same frozen inputs1, store each committed answer unchanged2, and rate it only once the world has graded it3.
- 1Frozen inputs. Each model gets the same snapshot: prices up to a closing date, or a world’s rules without its physics. Nothing newer.
- 2Committed answers. Models commit to portfolios, forecasts or controllers before anything is known. We store every answer unchanged.
- 3The world grades. Markets move over months; rovers drive unseen courses. Ratings come from those results, always with an interval.
Four verifiers
Checking an answer is radically easier than finding one. Every arena hands the checking to something the model cannot see or influence.
Why these four →“We, the inferior intelligence, the pupil, will have to evaluate a superintelligence, the teacher, while still guaranteeing objectivity. This problem, how an inferior intelligence can rigorously assess a superior one, I call the Kardashev problem.”
John Kinloch, Benchmarking in the age of superintelligence
The world is the verifier
Open-ended quests. Objectives, not questions and answers: the world grades the result, and the better the outcome, the better the model.
Markets · liveThe rules of a game is the verifier
Competition. Like chess, the game decides who is better even when every player is beyond us, and Elo keeps a statistically honest scoreboard.
Sandbox · livePhysics is the verifier
Complex engineering. The task is bounded by physics and chemistry, and nature does the grading: a lighter structure that still stands, a material with the property asked for.
Structures · in preparationLogic is the verifier
Formal proof. Mathematics written in a language a computer can check, such as Lean: an improved bound or a settled conjecture is verified, not marked.
Formal proof · plannedInsights
All insights →- What the models do with moneyAcross 273 portfolios, only high-risk ones are beating SPY. The models load up on gold, short Treasuries and chip-equipment makers, and almost never buy the biggest tech stocks.
- When a model thinks past its budgetIn the 6 October sandbox world, MiMo V2.6 Pro spent all 100,000 output tokens reasoning on one attempt and returned no code.
- Correcting our ratingsA bug in how we fitted Elo ratings put DeepSeek Flash first in Markets. It was last. Here is what went wrong and what we changed.