Object Synthesis by Learning Part Geometry with Surface and Volumetric Representations

We propose a conditional generative model, named Part Geometry Network (PG-Net), which synthesizes realistic objects and can be used as a robust feature descriptor for object reconstruction and classification. Surface and volumetric representations of objects have complementary properties of three-dimensional objects. Combining these modalities is more informative than using one modality alone. Therefore, PG-Net utilizes complementary properties of surface and volumetric representations by estimating curvature, surface area, and occupancy in voxel grids of objects with a single decoder as a multi-task learning. Objects are combinations of multiple parts, and therefore part geometry (PG) is essential to synthesize each part of the objects. PG-Net employs a part identifier to learn the part geometry. Additionally, we augmented a dataset by interpolating individual functional parts such as wings of an airplane, which helps learning part geometry and finding local/global minima of PG-Net. To demonstrate the capability of learning object representations of PG-Net, we performed object reconstruction and classification tasks on two standard large-scale datasets. PG-Net outperformed the state-of-the-art methods in object synthesis, classification, and reconstruction in a large margin. [Display omitted] •Learning surface and volumetric geometry for more effective model training and shape prediction.•Synthesizing objects with the parametric model given conditional information.•Developed a feature descriptor for shape classification and reconstruction.•Learning shape distribution with a conditional generative model with multi-task learning.