Google DeepMind leads $10M fund to study multi-agent AI risks

A $10 million research fund may look like a rounding error at a lab that spends billions annually, but the intent behind it is more pointed: even the teams building the most capable agent systems admit they do not know what happens when those systems interact at scale.

MIT Technology Review reports that Google DeepMind is anchoring a coalition to fund research on multi-agent artificial intelligence safety. Rohin Shah, who leads AGI safety and alignment work at the lab, put the problem bluntly: there is no established field of research for multi-agent safety, and the lab wants one to exist.

The coalition includes Schmidt Sciences, the philanthropic foundation set up by Eric and Wendy Schmidt; ARIA, the UK government's moonshot research agency; the Cooperative AI Foundation; and Google.org. The breadth of partners matters as much as the dollar figure. Academic groups, government agencies, and philanthropy are all being enlisted to work on a problem that industry labs have limited structural incentive to study at arm's length.

The architecture of the problem

Shah's concern is not about individual agents misbehaving. It is structural: agents increasingly receive instructions from other agents rather than from humans directly, creating delegation chains that no single actor fully monitors. Shah told MIT Technology Review that the analogy to human institutions is instructive. Complex institutions can accomplish things no individual can -- but they also fail in complex, emergent ways.

His tipping-point estimate is sobering: a few months, not years, before agent deployments reach the scale where these risks become concrete concerns. Google DeepMind itself accelerated that timeline. Agent-based tools were the centrepiece of Google I/O last month, meaning the lab is actively building toward the very scenario it is now funding others to study.

The strategic rationale for routing money outside the lab is explicit. Shah argues that academia can look further into the future than product-focused industry teams. The $10 million is designed to seed a field from scratch, not to fund the incremental safety work that labs would do regardless.

Evaluation as the missing layer

The timing sits alongside a broader industry reckoning with agent evaluation. Also this week, Infoworld reported that Microsoft open-sourced ASSERT, a framework that converts written specifications into executable tests for enterprise agents. Gartner's Anushree Verma noted that 99 percent of organizations deploy artificial intelligence agents into production without any formal pre-deployment evaluation. That figure applies to single-agent systems. Multi-agent interaction adds an entirely different dimension of complexity on top of an already unaddressed gap.

On the infrastructure side, Diagrid this week shipped Dapr 1.18 with what it calls verifiable execution: cryptographic proofs of how an agent ran, who held custody of a workflow, and whether execution records were altered, as SiliconAngle reported. That tooling addresses accountability after the fact. What Shah's fund is trying to build sits earlier in the pipeline -- theory and methods for anticipating how collections of agents will behave before anyone deploys them.

The gap between those two efforts reflects the current state of the field. Auditing tools are advancing rapidly. The science for predicting emergent behaviour in multi-agent systems, the kind of work that would inform both design and policy, is still largely absent from the artificial intelligence review literature.

What the money is actually supposed to buy

For practitioners building multi-agent pipelines today, formal safety evaluation methods simply do not exist at the required scale. The $10 million will not change that in the near term. What it can do is establish multi-agent safety as a legitimate research domain -- with funding lines, publication venues, and academic careers attached to it. That legitimacy is the prerequisite for any serious methodological progress.

Whether a few months is enough time to develop even baseline theory before agent-to-agent interaction reaches the scale that makes the risks real is a question the fund does not answer. Shah, for his part, seems to think the window is narrow.

---

FAQ

What is multi-agent AI safety and why does it matter now?
Most AI safety research focuses on individual models. Multi-agent safety studies what happens when many autonomous agents interact, delegate tasks to each other, and produce outcomes no single agent or human intended. It matters now because commercial agent deployments are scaling fast.

Who is in the $10 million multi-agent safety research coalition?
Google DeepMind, Schmidt Sciences, ARIA (the UK government's research agency), the Cooperative AI Foundation, and Google.org.

How is multi-agent risk different from single-agent risk?
Single-agent risk is largely about model behaviour in isolation. Multi-agent risk is about emergent system behaviour: feedback loops, cascading failures, and goal misalignment that only appear when agents interact at scale.

Are there existing tools for evaluating AI agents before deployment?
Few organizations use them formally. Microsoft's newly open-sourced ASSERT framework and platforms like LangSmith and Braintrust address single-agent evaluation. No widely adopted method yet exists for stress-testing multi-agent interactions at production scale.