LPMNet: Latent part modification and generation for 3D point clouds

•We propose a single end-to-end model for part-based 3D model generation.•It can handle generation and modification of both semantic parts and global shapes.•It can process unannotated point clouds with a segmentation module.•It supports varying sizes and numbers of parts with different resolutions. [Display omitted] In this paper, we focus on latent modification and generation of 3D point cloud object models with respect to their semantic parts. Different to the existing methods which use separate networks for part generation and assembly, we propose a single end-to-end Autoencoder model that can handle generation and modification of both semantic parts, and global shapes. The proposed method supports part exchange between 3D point cloud models and composition by different parts to form new models by directly editing latent representations. This holistic approach does not need part-based training to learn part representations and does not introduce any extra loss besides the standard reconstruction loss. The experiments demonstrate the robustness of the proposed method with different object categories and varying number of points. The method can generate new models by integration of generative models such as GANs and VAEs and can work with unannotated point clouds by integration of a segmentation module.