IMWA: Iterative Model Weight Averaging benefits class-imbalanced learning

Model Weight Averaging (MWA) enhances model performance by averaging weights of multiple trained models. This paper shows that MWA (1) is beneficial for class-imbalanced learning, (2) with early-epoch averaging yielding the most improvement. Building on these insights, we propose Iterative Model Weight Averaging (IMWA) for class-imbalanced learning tasks. IMWA divides training into multiple episodes, within which multiple models are trained from the same initial weights and then averaged into a single model. This averaged model initializes the next episode, creating an iterative approach. IMWA offers higher performance improvements compared to MWA. Notably, several class-imbalanced learning methods use Exponential Moving Average (EMA) to gradually update models weight for improving performance. Our IMWA method synergizes effectively with EMA-based approaches, leading to enhanced overall performance. Extensive experiments validate IMWA’s effectiveness across various class-imbalanced learning tasks, including classification and object detection. •We find that vanilla MWA performs well for class-imbalanced tasks and that early-epoch averaging yields greater gains, inspiring the design of IMWA.•IMWA iteratively conducts parallel training and weight averaging, and its integration with EMA shows their complementary benefits.•Extensive experiments demonstrate that IMWA outperforms vanilla MWA and effectively boosts performance for class-imbalanced learning.