Batch-level Experience Replay with Review for Continual Learning

Artificial intelligence systems often struggle to learn new information without losing what they already know, a problem that limits their real-world applications. Researchers from the University of Toronto and LG Sciencepark have developed a method called Experience Review Learning, which addresses this issue by enabling AI to retain and build upon past knowledge efficiently. This advancement, which secured first place in a global competition with 79 teams, could make AI more adaptable for tasks like autonomous driving or personalized assistants, where continuous learning is essential.

The key finding is that this approach significantly boosts AI performance in continual learning scenarios, where models process data in sequences rather than all at once. In tests using the CORe50 dataset, which includes images of domestic objects under varying conditions, the method improved average accuracy on validation sets—for instance, achieving up to 96.7% in the New Instances scenario, compared to much lower baselines. This means AI can now handle changing environments more reliably, reducing errors in tasks like object recognition.

Methodologically, the team built on experience replay, a technique where AI stores past data samples in a memory buffer. Unlike traditional methods that retrieve and combine data frequently, their modified approach retrieves samples only when new batches arrive and adds a 'review' step to remind the model of previously learned information. This reduces computational steps and enhances efficiency. They used a pre-trained DenseNet-161 model, fine-tuning it with stochastic gradient descent and incorporating data augmentations like flipping and brightness adjustments to improve generalization.

Results from the paper show clear improvements across different scenarios. In the New Instances setup, where batches have similar classes but different backgrounds, the method increased validation accuracy by 7.7% over basic fine-tuning. For New Classes, where each batch introduces entirely new categories, accuracy jumped from near-zero baselines to 59.4% with preprocessing. The data, referenced in tables like Table 3, indicate that the review component helps mitigate forgetting, especially in early training stages where accuracy would otherwise drop sharply.

In practical terms, this research matters because it brings AI closer to human-like learning, where knowledge accumulates over time without frequent retraining. For everyday users, this could lead to smarter devices that adapt to new information—like a security camera that learns to recognize new objects without forgetting old ones, or educational software that personalizes lessons based on student progress. The efficiency gains also mean lower computational costs, making such systems more accessible.

Limitations noted in the study include the method's reliance on specific datasets like CORe50, which may not fully represent all real-world variations. Additionally, in scenarios with highly dissimilar data batches, performance can still lag, indicating that further work is needed to handle extreme distribution shifts. The paper does not explore long-term learning over extended periods, leaving questions about scalability and durability in more complex environments.