AI Forgets Without Relearning

Machine learning systems often need to forget specific information—whether for privacy, compliance, or correcting errors—but traditional methods require costly retraining. Researchers from Nanyang Technological University have developed MPRU, a modular approach that enables AI models to unlearn targeted data without retraining, achieving similar performance at a fraction of the computational cost. This breakthrough addresses scalability and deployment challenges in real-world systems, making machine unlearning practical for diverse applications.

The key finding is that MPRU effectively removes specific classes or data points from a trained model by appending a projection-redistribution layer. This layer projects the model's output vectors onto a hyperplane orthogonal to the forgotten class's average representation, then redistributes the probability mass to remaining classes. The process requires only the model's outputs, not access to internal parameters or training data, ensuring minimal system alterations. Experiments on image datasets (CIFAR-10, CIFAR-100) and tabular data (Covertype) show that MPRU maintains accuracy close to full retraining, with forget accuracy differences as low as 0.0034 on CIFAR-100 and retain accuracy differences within 0.1322 on Covertype.

Methodologically, MPRU operates in two steps: projection and redistribution. First, it projects the output vector to remove influence from the forgotten class, using an orthonormal basis derived from class averages. Second, it redistributes the removed probability to other classes based on their original proportions, ensuring the output sums to one. This approach builds on an inductive view of classification, treating learning as a reversible sequence. The framework is model-agnostic, tested with CNN-based ResNet-18 for images and tree-based XGBoost for tabular data, and integrates seamlessly into serving pipelines like TensorFlow Serving without structural changes.

Results analysis reveals strong performance across metrics. On CIFAR-10, MPRU achieved KL divergence values between 0.1260 and 0.2238 for retained data and 0.6744 to 0.9495 for forgotten data, indicating close distributional similarity to retrained models. Mean squared error (MSE) was low, with at least 72% of data points showing deviations below 0.0145 on CIFAR-10, 0.0017 on CIFAR-100, and 0.013 on Covertype. Runtime comparisons highlight efficiency: MPRU took 0.00882 seconds on CIFAR-10 versus 200.66 seconds for retraining, a reduction of over 99.99%. Similar gains were seen on CIFAR-100 (0.08412 vs. 442.70 seconds) and Covertype (0.01698 vs. 12.84 seconds), confirming the method's low complexity, which scales as O(N C(M) + O(n^2)) for N test points and n classes.

In context, MPRU's practicality stems from its modularity and minimal requirements. It avoids the need for data or model access during unlearning, making it suitable for environments with restricted resources or privacy concerns. For general readers, this means AI systems can be updated to forget sensitive information—like personal data in healthcare or biased examples in hiring algorithms—without expensive retraining. This enhances compliance with regulations like GDPR, which mandate data deletion, and supports adaptive AI in dynamic settings such as autonomous vehicles or financial modeling, where models must quickly discard outdated or incorrect knowledge.

Limitations, as noted in the paper, include potential accuracy drops in some cases, such as for class 53 in CIFAR-100, where redistribution blurred class separations. The method assumes high model accuracy (at least 0.8) and low output variance for reliable projection, which may not hold in noisier datasets. Additionally, unlearning performance varies by class, with forget KL divergence reaching up to 26.2583 on CIFAR-10, indicating that some classes are harder to erase completely. Future work could address these issues by refining the redistribution step or extending the approach to more complex data types.