Multi-task multi-objective evolutionary network for hyperspectral image classification and pansharpening

•A multi-task multi-objective evolutionary network is proposed.•The framework combines two tasks by effective high-frequency information sharing.•Multi-task learning is modeled as a deep MOEAD to obtain trade-off solutions.•The method provides diverse and sufficient samples for multi-task. Multi-task learning has commonly been used and performed well at joint visual perception tasks. Hyperspectral pansharpening (HP) and hyperspectral classification (HC) tasks extract high-frequency information to enhance edges and classify samples, offering potential for performance improvements in multi-task learning. However, differences between tasks can make it challenging to balance their performances. To address this challenge, this paper proposes a multi-task multi-objective evolutionary network (DMOEAD) for joint learning of HC and HP. A multi-task sufficiency-and-diversity sampling method is designed to unify the heterogeneity of sample construction between two types of tasks. Two types of task-specific networks are constructed to decompose high-frequency information. Further, a collaborative learning module is designed to dynamically learn complementary high-frequency information from another task in different layers. To be compatible with the optimization direction of two types of tasks, multi-task optimization is realized using a deep multi-objective evolutionary algorithm (DMEO). In the DMEO, the set of parameters of the DMOEAD is regarded as an individual. A deep mutation operator is designed and used for network optimization, which accelerates large-scale network parameter searching. The DMEO can coordinate the differences between multiple tasks and provide a set of Pareto network parameter solutions. Finally, the experimental results demonstrate that the proposed method can significantly enhance the performance of both pansharpening and classification tasks.