A field that doesn't exist yet

Almost every dollar spent on AI safety so far has gone toward making one model behave: not lie, not leak, not help build a weapon. DeepMind, Schmidt Sciences, ARIA, the Cooperative AI Foundation and Google.org have now put up to $10M behind a different question — what a single well-behaved agent does when it's dropped into a crowd of millions of others, run by companies and people who don't coordinate and don't trust each other. The program is called Scaling AI Safety for a Multi-Agent World, and the striking thing is how candidly it's pitched.

Shah's estimate of the window to study multi-agent safety before it lands at deployment scale: a few more months.

"[T]here just isn't really a field of research for multi-agent safety yet," DeepMind's AGI-safety lead Rohin Shah told MIT Technology Review. "And we would like there to be." That is a frontier lab spending real money not on a result but on manufacturing an entire discipline from near-zero — grants of up to $300K and $1M, a deadline this August, awards in the autumn, open to academics and independent researchers worldwide.

The risks named are less exotic than they sound: scams, prompt injections that turn a helpful agent into self-guiding malware, cyberattacks — mostly familiar internet problems, but ones that could compound in ways no single-agent analysis would catch. The bet is that this failure mode arrives with deployment scale, and that the window to study it before it lands is, in Shah's estimate, a few more months. Whether a field can be summoned into being that fast is now a $10M experiment.