AI Finds Hidden Evidence by Asking Multiple Questions

Searching through complex documents like technical manuals, pharmaceutical reports, and textbooks for specific evidence—such as a crucial table, diagram, or procedure—has long been a for artificial intelligence. These visually rich documents often contain layouts where the key information is not in plain text but embedded in figures or schematics, and user queries may use different terminology than the document itself. A new approach called LITTA (Late-Interaction and Test-Time Alignment) addresses this by making AI ask multiple, slightly different versions of a question to find the right page, significantly improving retrieval accuracy without needing to overhaul existing search systems. This is particularly impactful for domains like industrial maintenance, where evidence is scattered and terminology varies, offering a practical upgrade for real-world applications.

The core finding from the research is that generating multiple query variants—alternative phrasings of the same question—consistently improves the retrieval of relevant evidence pages across diverse document types. In experiments on three domains—computer science textbooks, pharmaceutical reports, and industrial maintenance manuals—multi-query retrieval outperformed single-query baselines on key metrics. For example, as shown in Table 1, using three query variants (Q=3) increased the average NDCG@10 score from 64.2% to 69.5%, with the largest gains in industrial manuals, where accuracy jumped from 39.05% to 53.67%. This indicates that asking multiple questions helps AI overcome terminology mismatches and locate evidence that might be missed with a single query, especially in visually dense materials like schematics and procedural tables.

Ology behind LITTA is designed for simplicity and compatibility with existing systems. Given a user query, a large language model generates a small set of complementary variants—typically three to five—that preserve the original intent but use alternative terminology, synonyms, or domain-specific labels. Each variant is then used to retrieve candidate pages from a pre-computed index of document embeddings using a frozen vision retriever, specifically Nemotron ColEmbed V2, which employs late-interaction scoring. This scoring , detailed in the paper, compares query tokens against visual tokens from page images, allowing fine-grained matching to tables and diagrams. The ranked lists from all query variants are aggregated using Reciprocal Rank Fusion, a robust technique that prioritizes pages consistently ranked highly across variants, as illustrated in Figure 1 of the paper.

Analysis of reveals clear benefits in evidence coverage and ranking quality. Table 2 shows that multi-query retrieval improves recall metrics, with Recall@10 increasing from 75.30% to 78.45% on average when moving from one to three query variants, indicating that more relevant pages are retrieved. Figure 2 further demonstrates that top-k accuracy curves rise consistently with more variants, particularly in the industrial domain, where early retrieval success improves markedly. The paper notes that gains are most pronounced in settings with high visual and semantic variability, such as industrial manuals, where evidence is often encoded in schematics with heterogeneous naming conventions. In contrast, domains like computer science textbooks show smaller but still positive improvements due to more consistent terminology.

Of this research are significant for practical applications where accurate document retrieval is critical, such as in technical support, regulatory compliance, or educational resources. By improving evidence page retrieval without retraining retrievers, LITTA offers a cost-effective way to enhance existing multimodal search systems, making them more robust to user phrasing variations. 's controllable accuracy–efficiency trade-off, where performance scales with the number of query variants, allows deployment under latency constraints, as noted in the paper's complexity analysis. This could benefit industries relying on precise manual lookups, reducing errors and saving time in information retrieval tasks.

However, the approach has limitations. Multi-query retrieval increases online computational cost roughly linearly with the number of variants, which may affect latency in time-sensitive applications. The quality of query expansions also depends on the language model used, and less faithful variants could introduce noise, though the paper reports diminishing returns beyond a small number of variants. Additionally, assumes pre-computed page embeddings and does not address cases where evidence spans multiple pages or requires deeper reasoning beyond retrieval. Future work, as suggested in the paper, could explore adaptive strategies for query generation or incorporate lightweight re-ranking to further refine .