AI Can Now Verify Long Documents in Real Time

In enterprise settings where AI assists with legal contracts, financial reports, or scientific papers, ensuring that generated answers are accurate and supported by source documents is critical. However, current verification systems often fail because they can only check small portions of long documents, missing crucial evidence buried deep within the text. This limitation poses real risks in compliance-sensitive areas like contract analysis or clinical reporting, where unsupported claims could lead to operational errors. A new approach addresses this by enabling real-time verification across full documents, balancing speed and thoroughness to make AI tools more trustworthy for practical use.

The researchers developed a system that can verify AI-generated responses against entire documents up to 32,000 tokens long, a significant improvement over typical s limited to 8,000 tokens. This full-context verification dramatically improves the detection of unsupported or hallucinated content. For example, in tests on long documents averaging 17,500 tokens, the new increased hallucination recall by 817% compared to truncated verification, meaning it catches far more errors that shorter-context models would miss. This capability is essential because in real documents, such as contracts or regulatory filings, key evidence often appears far from the beginning, and partial checks can silently overlook it.

To achieve this, the team extended an encoder-based verification model using a technique called Rotary Position Embedding (RoPE) scaling, which allows the model to handle longer sequences. However, simply enlarging the context window wasn't enough; they had to prevent the model from forgetting how to pay attention to distant parts of the text. They used retrieval-aware masking strategies during training, which force the model to rely on information from earlier in the document to predict masked tokens, preserving long-range sensitivity. Additionally, they applied Elastic Weight Consolidation regularization to avoid overwriting pre-trained knowledge, ensuring the model remains effective for both short and long documents.

Show that the extended model maintains performance on short documents, with token F1 scores comparable to baseline 8K models, while excelling on long ones. In a benchmark constructed from long documents like stories, articles, and government reports, the full-context verifier achieved a hallucination F1 of 0.50, a 400% improvement over the 8K model's 0.10. The researchers also introduced an early-exit inference framework, where lightweight adapters at intermediate layers allow the system to trade some accuracy for speed. For instance, exiting at layer 16 instead of the full depth (layer 22) retains 92.8% of the example F1 accuracy but speeds up processing by 1.4 times, making it suitable for real-time applications.

This advancement has significant for industries relying on document-heavy AI assistants. By enabling verification across entire documents without sacrificing latency, it reduces the risk of errors in critical tasks like legal review or financial analysis. The configurable early-exit feature allows organizations to tailor the system to their needs—opting for faster checks in interactive services or more thorough analysis in offline settings. As AI becomes more integrated into enterprise workflows, such reliable verification s are crucial for building trust and ensuring compliance, potentially preventing costly mistakes in high-stakes environments.

Despite its strengths, the approach has limitations. The current system supports up to 32K tokens, but many real-world documents, such as regulatory filings, can exceed 100K tokens, requiring further extension. The researchers note that both their and naïve approaches struggle with very long separations beyond 4K tokens, indicating that fine-tuning dynamics, not just architectural changes, limit extreme long-context capabilities. Future work aims to extend support to 64K–128K tokens, develop confidence-aware dynamic inference for lower latency, and create multilingual benchmarks to improve cross-domain reliability. These steps will help address the growing demand for robust verification in increasingly complex document sets.