AIResearch AIResearch
Back to articles
Science

Why Systems Stay Broken on Purpose

A new game theory model reveals how adversaries profit from keeping systems inefficient, explaining everything from social media polarization to political gridlock.

AI Research
November 14, 2025
3 min read
Why Systems Stay Broken on Purpose

Many systems, from social networks to political institutions, remain stuck in suboptimal states that seem irrational. A new study explains this 'rational stagnation' as a stable equilibrium maintained by adversaries who benefit from inefficiency. The researchers introduce a game-theoretic framework where an adversary's utility comes from the gap between a system's ideal performance and its actual state, a principle they term 'loss.' This model shows how stagnation is not a failure but a strategically maintained outcome, with implications for understanding polarization, misinformation, and institutional design.

The key finding is that rational adversaries can sustain stagnation by manipulating system parameters to keep it in a 'fragile band' where cooperation and conflict coexist. Starting with the Prisoner's Dilemma, the researchers transform players' utilities using parameters like the 'ratio of recognition' (w = b/a), which measures interpersonal identification. They identify critical thresholds (w_min and w_max) that define phases: distrust (w < w_min), fragile cooperation (w_min ≤ w ≤ w_max), and full cooperation (w > w_max). In the fragile band, both cooperative and defective equilibria exist, allowing adversaries to maintain instability without collapse.

Methodologically, the study extends this to a dynamic model with stochastic cooperative payoffs (R_t) and intervention costs (C_c for collapse, C_m for maintenance). Using a generalized Bellman equation, the researchers derive three strategic regimes for adversaries: immediate destruction (when gains are low), rational stagnation (when maintaining fragility yields higher long-term losses), and abandonment (when costs exceed benefits). Under ideal conditions with zero costs and non-decreasing R_t, stagnation occurs if the growth rate of the potential loss exceeds the discount rate, making delay optimal.

Results analysis, referencing figures and equations from the paper, shows that in the fragile band, systems exhibit probabilistic tipping points rather than sharp thresholds. For example, in social media, algorithms maximize engagement by keeping user interactions in this band, avoiding full cooperation (which reduces engagement) or total collapse (which risks revenue loss). The model predicts that such algorithms neither eliminate misinformation nor cause network failure, as stagnation balances profitability and risk. In political meta-games, external adversaries intervene to keep rival societies in stagnation, weakening their capacity without inducing destruction.

In context, this matters because it reframes common issues like polarization and gridlock as outcomes of rational strategies, not irrational behavior. For AI and institutional design, it implies that systems optimizing for metrics like engagement may inherently develop adversarial traits, maintaining fragility to maximize objectives. This perspective shifts the focus from fixing malfunctions to designing rules that reduce the profitability of stagnation for adversaries.

Limitations from the paper include the normative definition of the 'ideal' state, which can be arbitrary, and assumptions about instantaneous changes in the actual state, ignoring inertia. The model also requires clear empirical calibration of variables like the ratio of recognition and intervention costs to ensure testability. Future work could explore structural adversaries who manipulate game rules directly, extending the framework to broader institutional analyses.

About the Author

Guilherme A.

Guilherme A.

Former dentist (MD) from Brazil, 41 years old, husband, and AI enthusiast. In 2020, he transitioned from a decade-long career in dentistry to pursue his passion for technology, entrepreneurship, and helping others grow.

Connect on LinkedIn