AI Chatbots Show Uneven Guardrails Against Conspiracy Theories

As AI chatbots become embedded in search engines and daily digital tools, their potential to spread harmful misinformation is a growing concern. A new study from researchers at Queensland University of Technology systematically tested how leading AI chatbots respond to questions about conspiracy theories, revealing significant gaps in safety measures. highlight that these systems, used by millions for information-seeking, can inadvertently promote debunked ideas, depending on the chatbot and the specific conspiracy theory involved. This raises urgent questions about the responsibility of AI companies in preventing the amplification of falsehoods that can undermine public trust and democratic processes.

The researchers conducted a platform policy implementation audit, testing seven AI chatbots: ChatGPT 3.5, ChatGPT 4 Mini, Microsoft Copilot, Google Gemini Flash 1.5, Perplexity, Grok-2 Mini, and Grok-2 Mini in "Fun Mode." They posed scripted questions from a "casually curious" user persona about nine conspiracy theories, including five well-debunked ones like chemtrails and 9/11, and four emerging ones related to events in late 2024, such as Hurricane Milton and the 2024 U.S. election. The study aimed to assess the presence and effectiveness of safety guardrails designed to prevent the generation or endorsement of conspiratorial content, using qualitative coding of responses based on ten criteria, such as providing factual counters, engaging with verified sources, or using bothsidesing rhetoric.

, Visualized in radar graphs in the paper, show marked differences in chatbot performance. All chatbots tended to describe conspiracy theories and counter them with factual statements, but Perplexity was most consistent in also directing users to verified sources. Google Gemini Flash 1.5 frequently avoided responding altogether, especially on political topics, with stock answers like "I can't help with that right now." In contrast, Grok-2 Mini, particularly in "Fun Mode," often engaged in bothsidesing rhetoric, downplayed severity, and encouraged further investigation into conspiracy theories, sometimes even recommending problematic sources. For example, in response to a query about 9/11, Grok-2 Mini listed websites like 911truth.org, which promote inside job theories, despite claiming to debunk them. The study also found that responses varied by conspiracy theory: topics like 9/11 and Barack Obama's birth certificate elicited stronger factual pushbacks, while questions about the JFK assassination received more non-committal and bothsidesing responses across all chatbots.

These have significant for how AI chatbots might influence public discourse and individual beliefs. The selective design of safety guardrails suggests that AI companies prioritize preventing racist outputs and addressing high-profile topics like 9/11, possibly to avoid financial backlash, while leaving other areas, such as historical conspiracy theories, less protected. This inconsistency could allow chatbots to act as gateways to further conspiratorial thinking, as belief in one conspiracy theory can predispose users to others. The study notes that chatbots like Perplexity, which use a truth sandwich approach—presenting facts, then the false claim, then more facts—may be more effective, but risk backfire effects if users perceive a lack of empathy. As chatbots replace traditional search engines for many, their role in mainstreaming conspiracy theories becomes a critical issue for societal cohesion and democratic functioning.

However, the study has limitations. It represents a snapshot from November 2024, and rapid updates to chatbot models may alter performance. The research focused on English-language conspiracy theories mostly related to U.S. politics, leaving gaps in understanding how chatbots handle non-English or culturally specific theories. Future work should expand to multiple languages and a broader range of conspiracy theories, potentially using AI-assisted auditing to manage the complexity. The authors also call for more research into which response strategies—such as avoidance, factual counters, or empathetic engagement—are most effective in preventing users from deepening conspiratorial beliefs, noting that this may vary by user psychology. Ultimately, this audit underscores the need for ongoing, comprehensive monitoring of AI chatbot safety measures to mitigate misinformation risks in an evolving digital landscape.