AI Can Now Read Your Mind While Driving

When you drive, you don't see everything around you equally. Your attention focuses on what matters—the car ahead, not the parked vehicle on the side. This selective attention shapes how you navigate the world, and until now, artificial intelligence systems interacting with humans largely ignored this fundamental aspect of human cognition. Researchers from New York University and Toyota Research Institute have developed a method that allows AI to estimate these attentional biases from observed behavior, bridging a critical gap between how people actually make decisions and how machines model them.

The key finding is that AI can now infer what people pay attention to during goal-directed tasks by analyzing their actions. The researchers demonstrated that their attention-aware inverse planning approach systematically differs from standard methods that assume people act optimally with complete information. In driving scenarios, this means AI can determine whether a driver tends to ignore ice patches, cones, or other cars based solely on their driving behavior.

Methodologically, the team built on recent cognitive science showing that human decision-making is resource-limited. They extended the value-guided construal framework, which models how people form simplified mental representations of tasks. In their approach, a person's behavior reflects both their goals and their attentional biases, formalized through a heuristic function that weights different objects in the environment. The researchers used maximum likelihood estimation to infer these bias parameters from observed state-action sequences.

Results from two experimental settings validate the approach. In a tabular grid world simulating driving (Figure 1), the method robustly recovered biases toward ignoring cones (R² = 0.83) and ice patches (R² = 0.76), though estimating biases about parked cars was more challenging (R² = 0.20). In high-fidelity simulations using the GPUDrive simulator and Waymo Open Motion Dataset, the algorithm reliably inferred heuristic weights related to hazard perception, such as deviation from ego heading and relative collision risk, across 10 real-world driving scenes (Figure 5). Crucially, the researchers showed that standard inverse reinforcement learning methods fail to capture attention-limited behavior even when they fit the data reasonably well (Figure 3).

This work matters because AI systems that misunderstand human attention can lead to dangerous interactions. In autonomous driving, a vehicle that assumes human drivers see everything might make unsafe decisions during merges or lane changes. More broadly, any AI that collaborates with humans—from assistive robots to industrial systems—becomes safer and more effective when it accounts for how people actually perceive and prioritize information. The approach scales to complex, continuous domains like driving, suggesting practical applications in real-time human-AI interaction.

Limitations include the assumption that attentional biases remain fixed throughout a task, whereas people dynamically shift focus in response to events. The current model also focuses exclusively on attentional biases, ignoring other cognitive factors like time pressure or perceptual noise. Future work could incorporate updating construals and joint inference of rewards and biases, particularly since the paper shows that misspecifying human decision processes leads to systematic errors in AI behavior.