Multi-prototype Relational Network for Few-Shot ALS Point Cloud Semantic Segmentation by Transferring Knowledge from Photogrammetric Point Clouds

Existing airborne laser scanning (ALS) point cloud semantic segmentation approaches are limited by their overreliances on sufficient point-wise annotations that further confine their generalization ability to new scenes. To overcome these problems, a novel three-stage multi-prototype relational network (Thr-MPRNet) is proposed for few-shot ALS point cloud semantic segmentation by transferring knowledge from well-annotated photogrammetric point clouds. In MPRNet, a 3D few-shot learning structure containing a feature learner and a relation learner is built to learn meta-knowledge from multiple point-wise tasks, and a multi-prototype generator is designed to represent the semantic distribution of point clouds that can dynamically adapt to large-scale scenarios. Then, to transfer knowledge across different domains, MPRNet is trained in a unified framework with three task-based learning stages. Prior knowledge is first meta-learned from the source photogrammetric point clouds and then transferred to novel target datasets with a few labeled ALS point clouds. Finally, the MPRNet can be flexibly generalized to the unlabeled target ALS point clouds without further retraining from scratch. In the experiments, the SensatUrban dataset is used as the source photogrammetric point clouds, and two ALS point cloud datasets (ISPRS and DALES) are used to evaluate the few-shot semantic segmentation ability of the proposed method. The experiments demonstrate that Thr-MPRNet obtains promising generalization performance on different target datasets. More importantly, it outperforms supervised networks with 10% labeled samples. In summary, the proposed method achieves state-of-the-art cross-domain semantic segmentation performance and greatly alleviates the dependence on ALS point cloud annotations.