The forecaster you can rent

Two Mentat finds ago, the question was when AI forecasting would draw level with the best humans: the Forecasting Research Institute projected parity around November, and a separate benchmark caught models stubbornly refusing to revise their guesses as the world moved. FutureSearch, a startup run by Metaculus's former chief technologist, is the version of that thesis you can now buy. Its ensemble sits at the top of Metaculus's own live FutureEval tournament — first of 163 entrants, scored by an independent party — and on the Institute's ForecastBench it runs even with the human superforecaster median. Feed it a question like 'will US respiratory infections be cut in half by 2040' and it reads 212 sources in five minutes and returns 7 percent, landing between a rival AI at 8.8 and a human at 5 to 10.

the best human superforecasters in the world and the best bots are too close to clearly tell apart — Scott Alexander

The honest reading is a tie, not a rout. Scott Alexander, in the main independent write-up, calls the best humans and the best bots 'too close to clearly tell apart' — humans still took the top two spots in a recent Metaculus Cup with an AI third, and he pins some of the machines' showing on 'a heavy dose of luck.' The wins are also all near-term and in-distribution: the genuinely hard cases — long-horizon, society-wide, strange events — remain unsettled. The maker's cleaner 'beats the human median' line is technically true and quietly selective; on that same ForecastBench it ranks fourteenth of 259.

The more durable idea is why forecasting makes a good test at all. Most AI benchmarks get saturated and gamed because humans write the questions, and, as the company's founder puts it, the people building evals aren't smarter than the models anymore. Forecasting escapes that: reality grades the answer, so the supply of hard, un-cheatable questions is endless. Their trick for not waiting years to find out — feed a model an internet snapshot from before its training cutoff and ask it to 'forecast' what has, to us, already happened. It scores a new forecaster in a day instead of a season, and turns the unknowable future into a renewable exam the world marks for you.