AI Shields Text Recognition from Hidden Attacks

In today's digital world, optical character recognition (OCR) systems are everywhere, from scanning documents in offices to reading license plates on roads. These systems help automate tasks that would otherwise require human effort, but they have a hidden weakness: attackers can subtly alter text images in ways that are invisible to the human eye, causing the OCR to produce wrong transcriptions. This isn't just a theoretical problem—it can lead to real-world issues like financial errors or incorrect legal decisions when OCR is used in high-stakes applications. Existing defenses often struggle because they are tied to specific models, slow down processing, or fail against new types of attacks, leaving critical systems vulnerable. A new approach called TopoReformer tackles this by focusing on the underlying structure of images, using ideas from topology, the branch of mathematics that studies properties like connectivity and holes that don't change when shapes are stretched or bent. This aims to make OCR systems more reliable without needing constant updates or sacrificing performance on normal inputs.

The researchers behind TopoReformer discovered that by preserving the topological features of text images, they could effectively remove adversarial perturbations while keeping the text readable. Adversarial attacks work by making tiny changes to pixels that confuse AI models, but these changes often don't affect the overall shape and connectivity of the characters. For example, in tests on datasets like MNIST and EMNIST—which include handwritten digits and letters—improved the F1 score, a measure of accuracy, from as low as 4.30% without defense to 75.15% with TopoReformer under strong Carlini-Wagner attacks. This means that even when attackers used sophisticated techniques to hide changes, the system could still recognize the text correctly by focusing on the stable, global structure rather than getting tricked by local distortions.

To achieve this, the team developed a pipeline that starts with a topological autoencoder, a type of neural network trained to encode images in a way that emphasizes their topological properties. This autoencoder uses persistent homology, a tool from computational topology that tracks how features like connected components and loops appear across different scales in the data. By comparing the topology of the input image and its encoded version, the system ensures that essential structures are preserved. The output is then passed through a reformer module and an auxiliary component, which fine-tune the image to match what the OCR model expects. A key part of the training is the freeze-flow paradigm, where gradients are routed through the auxiliary module first to stabilize learning, leading to up to 5% better performance in some cases. Importantly, the entire system was trained only on clean, unperturbed data, making it easier to deploy without needing examples of attacks.

From extensive testing show that TopoReformer holds up well against a variety of threats. In experiments with classical attacks like FGSM and PGD, maintained high F1 scores—for instance, 97.70% on MNIST under PGD attacks, compared to 96.74% without defense. It also performed robustly against adaptive attacks designed to bypass defenses, such as EOT and BPDA, reducing attack success rates from nearly 100% to as low as 9.19% in some scenarios. For OCR-specific attacks like FAWA, which uses watermarks to mislead systems, TopoReformer cut the attack success rate significantly; for example, in the CRNN model, it dropped from 100% to 78.83%, while character accuracy rose from 48.13% to 71.00%. Visualizations with Grad-CAM, a technique that highlights where the model focuses in an image, confirmed that TopoReformer keeps attention on the relevant parts of the text, even when adversarial noise is present, leading to more confident and accurate predictions.

Of this research are broad, as it offers a model-agnostic defense that can be added to existing OCR systems without retraining them from scratch. This could enhance security in areas like document processing, where errors might cause compliance issues, or in automated traffic systems, where misread license plates could lead to unfair penalties. By relying on topological principles, the approach provides a generalizable way to handle unseen attacks, potentially saving time and resources compared to s that require constant adversarial training. However, the paper notes that the defense is less effective under BPDA attacks alone, suggesting that while it smooths out global distortions, it might not address all local vulnerabilities. Future work could integrate topological constraints directly into OCR models or create standardized benchmarks to further improve robustness in real-world applications.