Class-incremental learning with causal relational replay

In Class-Incremental Learning (Class-IL), deep neural networks often fail to learn a sequence of classes incrementally due to catastrophic forgetting, a phenomenon arising from the absence of exposure to old knowledge. To alleviate this issue, conventional rehearsal methods, such as experience replay, store a limited number of old exemplars and then interleave with the current data for joint learning and rehearsal. However, the networks following this training scheme might not successfully reduce forgetting due to the lack of direct consideration of relations between samples of previously learned and new classes. Drawing inspiration from how humans learn by noticing the similarities and differences between classes, we propose a novel Class-IL framework called Relational Replay (RR). RR learns and recalls relations between images across all classes over time. To ensure these relations remain intrinsic and robust to forgetting, we incorporate causal reasoning to RR, resulting in Causal Relational Replay (CRR). CRR analyzes these relations using a causality perspective, aiming to identify intrinsic relations rooted in the images’ semantic features, serving as the cause of these relations. Our proposed method shows a competitive performance compared to the state-of-the-art rehearsal methods in Class-IL with clear and consistent improvements in the majority of settings on standard benchmark datasets. •We tackle the problem of catastrophic forgetting in Class-IL.•Our method leverages object relations, inspired by human memory, to prevent forgetting.•We discover spurious relations worsen forgetting, best to focus only on intrinsic ones.•We add a causal learning mechanism to analyze relations and combat spurious factors.•Our algorithm excels in Class-IL, consistently surpassing top rehearsal techniques.