AI Learns to Reason Without Being Taught

Artificial intelligence systems often memorize data in ways that defy explanation, but new research reveals they can spontaneously develop reasoning abilities. A study from Carnegie Mellon University and Google Research shows that AI models, including Transformers and Mamba, learn to solve complex path-finding tasks without explicit training, uncovering a hidden geometric structure in their memory. This challenges the long-held view that AI simply stores associations, suggesting it builds internal models of relationships.

The researchers designed a path-star task where AI models had to navigate from a central node to a leaf in a graph, a problem requiring multi-step reasoning. When trained only on local edge connections, the models successfully predicted unseen paths in large graphs with up to 10,000 nodes, achieving 100% accuracy in some cases. This indicates the AI synthesized global knowledge from mere associations, learning to infer paths it never directly encountered.

To achieve this, the models were trained using next-token prediction on sequences representing graph edges, without any step-by-step guidance or chain-of-thought supervision. The setup involved memorizing individual connections and then applying this to path-finding, using a decoder-only Transformer architecture. Crucially, the models learned the hardest decision-making step—the first token in a path—in isolation, demonstrating an ability to handle compositional tasks that typically stump AI.

Analysis of the models' embeddings revealed a geometric memory structure, where node positions in a high-dimensional space reflect their relationships in the graph. For example, embeddings of leaf nodes clustered with their correct first-hop neighbors, enabling efficient path prediction. This contrasts with associative memory, which would require exponential computations for such tasks. The study found this geometry emerges naturally during training, even without architectural pressures favoring it.

This discovery has practical implications for improving AI's reasoning in real-world scenarios, such as navigation systems or data retrieval, where models need to connect disparate information. It also highlights potential limits in editing or unlearning knowledge due to the interconnected nature of geometric memory. However, the research is limited to symbolic tasks and specific graph topologies, leaving open how well it generalizes to natural language or other domains.

The findings suggest that current AI models have untapped potential for geometric reasoning, pointing to ways developers could enhance tasks like retrieval or creative problem-solving. By understanding how AI builds internal structures, researchers can design systems that better mimic human-like reasoning without exhaustive training.