AI Backdoor Attacks Hide in Compressed Data

Dataset condensation, a technique for creating compact versions of large datasets to save storage and computation, has become a popular tool in AI development. However, new research reveals a critical vulnerability: malicious actors can inject hidden backdoors into these condensed datasets, compromising downstream AI models without raising alarms. This highlights a significant security risk in data-sharing pipelines, where compressed data is often trusted as a safe alternative to raw information. The study introduces S NEAKDOOR, a that achieves a balance between attack effectiveness and stealthiness, making it particularly dangerous for real-world applications.

S NEAKDOOR operates by identifying the most vulnerable class pairs in a dataset—those with the highest inter-class misclassification rates—and then generating input-aware triggers tailored to individual samples. Unlike previous attacks that used fixed patterns, this approach creates perturbations aligned with the local features of each image, making them visually coherent and hard to detect. involves two main stages: trigger generation, where a generative network produces these subtle perturbations, and backdoor injection, where triggered samples are mixed into the target class during condensation. This allows the condensed dataset to encode malicious behavior while preserving its utility for normal tasks, as demonstrated across multiple datasets like CIFAR-10 and STL-10.

The researchers evaluated S NEAKDOOR against existing backdoor attacks, including Naive, Doorping, Simple, and Relax, using metrics such as Attack Success Rate (ASR), Clean Test Accuracy (CTA), and Stealthiness (STE). showed that S NEAKDOOR consistently achieved a superior balance, with high ASR (often above 97% on datasets like STL-10) and competitive CTA, while maintaining low detectability. For instance, on STL-10 with the DM condensation , S NEAKDOOR achieved an ASR of 0.973 and a CTA of 0.598, outperforming baselines in stealth metrics like PSNR and SSIM. 's effectiveness was validated across six datasets and various condensation techniques, including DM, DC, IDM, and DAM, with cross-architecture tests showing strong transferability to models like VGG11 and ResNet18.

This vulnerability has serious for AI security, especially in scenarios where condensed datasets are shared in privacy-sensitive or resource-constrained environments, such as federated learning or edge AI applications. If exploited, backdoored models could misclassify critical inputs—like road signs in autonomous vehicles or medical images—without obvious signs of tampering. The study's theoretical analysis, conducted in a Reproducing Kernel Hilbert Space, provides formal guarantees on stealthiness, showing that triggered samples remain close to the clean data manifold under bounded perturbations. This underscores the need for proactive defenses, such as robust anomaly detection and certified safeguards in condensation pipelines, to prevent such attacks from undermining trust in AI systems.

Despite its strengths, S NEAKDOOR has limitations. It does not consistently surpass all existing s on every single metric; for example, Doorping sometimes achieves higher ASR in isolation. The attack also relies on a relatively high poisoning ratio to reach optimal effectiveness, which may not be practical in all real-world settings. Additionally, it may struggle with more complex threat models, such as targeted manipulations involving specific object transformations. Future work could focus on reducing the poisoning requirement and extending the approach to handle diverse attack objectives, while serve as a cautionary tale for developers to implement rigorous provenance tracking and transparency in data condensation processes.