AI Can Now Check if Public Questions Are Fairly Represented

In online deliberative processes like citizens' assemblies and polls, participants often propose questions for expert panels, but only a few are selected due to time and space constraints. This raises a critical issue: do the chosen questions truly represent the diverse interests of all participants? Researchers have developed a computational framework to audit and improve the fairness of question selection, addressing a key challenge in scaling democratic engagement. Their work, applied to real-world deliberations across 50 countries, offers a practical tool for ensuring that public voices are not sidelined in policy discussions.

The key finding is that algorithmic methods can significantly enhance the representativeness of questions posed to experts. The researchers introduced a measure based on the concept of justified representation (JR), which ensures that no large group of participants with shared concerns is overlooked. In tests on historical deliberative polls, both an integer programming approach that selects the best subset of participant-proposed questions and large language model (LLM) methods that generate summary questions often outperformed human moderators in achieving higher JR values. For instance, in one deliberation on democratic reform, the human-selected slate had a JR value of 0.842, while the optimized integer programming slate achieved 0.077, indicating much better representation.

Methodology involved using text embeddings to infer how much participants value different questions, based on the similarity between their own proposals and other questions. This approach, validated with datasets like Quora Question Pairs, showed high accuracy in matching human judgments of question similarity. The team developed efficient algorithms to compute JR values, with the fastest running in O(mn log n) time, where m is the number of questions and n the number of participants. This made it feasible to apply the audit in real-time on an online deliberation platform used for hundreds of sessions.

Results analysis revealed that in 12 deliberative poll sessions—covering topics from democratic reform to AI policies—algorithmic selections consistently provided more representative question slates than random or human choices. For example, in the 'America One Room' deliberation, LLM-generated questions achieved JR values as low as 0.032 in some cases, compared to 0.842 for human selections. Cross-validation across different embedding models confirmed the robustness of these findings, with JR values remaining low and consistent, indicating that the improvements are not dependent on a specific AI model.

Contextually, this research matters because it enhances the integrity of democratic processes where public input shapes policy. By ensuring that expert panels address a broad spectrum of concerns, the method helps build trust and inclusivity in online deliberations. Practitioners can now use the integrated platform to audit and refine question selection in future events, making it easier to scale participatory democracy without sacrificing fairness. This is particularly relevant as digital tools become central to civic engagement, offering a way to balance efficiency with equitable representation.

Limitations include the reliance on text embeddings for utility inferences, which, while validated, may not capture all nuances of participant preferences without direct feedback. The study focused on JR as a measure, but other axioms like balanced justified representation (BJR) could offer stronger guarantees and warrant further exploration. Additionally, the retrospective analysis of historical data limits the ability to validate inferences with real-time participant responses, pointing to a need for ongoing studies in live settings.