Facing spatiotemporal heterogeneity: A unified federated continual learning framework with self-challenge rehearsal for industrial monitoring tasks

Federated learning (FL) has attracted significant interest in industrial monitoring tasks. Practically, FL-based methods face spatiotemporal heterogeneity, including the inter-client distribution discrepancy spatially and intra-client distribution change temporally. Some methods introduce continual learning into FL methods, which aims to facilitate the models to preserve historical knowledge. However, most of them ignore the interference between previous and collective knowledge caused by spatiotemporal heterogeneity, leading to inaccurate historical memories and unstable aggregation. This study proposes a unified federated continual learning framework for industrial monitoring tasks, which introduces an “intra-rehearsal and inter-aggregation” mechanism to address spatiotemporal heterogeneity. To recall historical distributions for continual learning capability, a self-challenge diffusion and replay model is proposed by designing masking recovery tasks with concurrent random scale and position. It is proven to reduce the conditional entropy and improve the ability to portray complex distributions. To reduce parameter variations during federated aggregation, a gradient balance policy is designed by applying iteration-related regularity to the cumulative gradient within rounds. It is theoretically analyzed to reduce the upper bound of the minimal global gradient. Our unified framework is applicable to supervised and unsupervised paradigms and thus covers typical industrial monitoring tasks. Experiments on a multiphase flow process evaluate the continual learning performance of the proposed framework. Compared to the sub-optimal algorithms, our ERR decreases by 9.20% and 10.60%, and FWT decreases by 0.039 and 0.030 in RMSE and MAE, respectively. In addition, the experiments are conducted to demonstrate the superiority of rehearsal ability and training stability.