AI Models Have Culture-Specific Brain Cells

Artificial intelligence systems that process images and text, known as vision-language models, are increasingly used in applications from visual question answering to document parsing. However, these models often struggle with culturally specific content, performing unevenly across different cultural contexts. A new study reveals that this limitation stems from specialized neurons within the models that respond selectively to cultural cues, offering a path to more equitable AI systems.

The researchers discovered that vision-language models contain neurons that activate preferentially when processing inputs tied to specific cultures. By identifying and deactivating these neurons, the team demonstrated that the models' performance on culturally grounded questions drops significantly for the targeted culture, while remaining largely unaffected for others. This finding provides empirical evidence that cultural knowledge is locally encoded in specific components of the neural network.

To identify these culture-sensitive neurons, the researchers developed a three-stage pipeline. First, they recorded neuron activations while the models answered culturally specific visual questions from the CVQA benchmark, which includes content from 25 cultural groups. They then scored neurons based on their selectivity using four methods: probability-based selection, entropy-based selection, mean activation difference, and a novel contrastive activation selection approach. Finally, they tested causality by deactivating top-ranked neurons and measuring the impact on model performance.

The contrastive activation selection method proved most effective, outperforming baseline approaches by focusing on the contrast between a neuron's responses to its most preferred culture versus its second-most preferred culture. This method identified neurons that, when deactivated, caused substantial performance drops specifically for their associated cultures with minimal spillover effects. For example, in Qwen2.5-VL-7B, deactivating CAS-identified neurons reduced accuracy by up to 5.52% for targeted cultures while affecting other cultures by less than 1%.

Analysis of where these culture-sensitive neurons reside in the network architecture revealed they tend to concentrate in mid-to-late layers of the decoder. In Qwen2.5-VL-7B, the researchers observed noticeable clusters around layers 6-8 and 15-18, suggesting these layers play a crucial role in culturally grounded reasoning. The distribution patterns varied across different model architectures, with Pangea-7B showing similar concentration patterns to Qwen2.5-VL-7B, while LLaVA-v1.6-Mistral-7B exhibited more diffuse encoding.

The identification of culture-sensitive neurons has significant implications for developing more fair and transparent AI systems. By pinpointing specific components responsible for cultural processing, researchers can develop targeted interventions to mitigate biases without retraining entire models. This approach could lead to sparse fine-tuning methods that selectively adjust only the relevant neurons, preserving overall model capabilities while improving cultural competence.

However, the study acknowledges several limitations. The research focused only on decoder MLP neurons, leaving other components like attention heads and encoders unexplored. The cultural taxonomy used from the CVQA benchmark, while practical for experimental feasibility, represents a simplification of real-world cultural diversity. Additionally, all experiments used English-translated prompts to isolate cultural knowledge from language proficiency effects, leaving open questions about how these findings extend to multilingual settings.

The work demonstrates that cultural knowledge in AI systems is not uniformly distributed but concentrated in specialized neural components. This insight opens new avenues for interpretability research and targeted model editing, moving toward AI systems that can better serve diverse global populations.