MuSAnet: Monocular-to-3-D Human Modeling via Multi-Scale Spatial Awareness

Monocular-to-3D human modeling involves creating colored three-dimensional models of humans from monocular try-on images. This technology offers personalized services to consumers and has garnered considerable attention for its potential business value. However, current methods are unable to deform clothing images to align with the human body naturally. Additionally, the generation of low-quality monocular try-on images severely hinders the creation of high-precision human models. This paper presents a novel monocular-to-3D human modeling network capable of accurately generating 3D models from monocular try-on images. To improve the accuracy of clothing deformation, an enhanced non-rigid deformation constraint strategy is introduced. This strategy helps reduce excessive deformation by strengthening penalties for outliers. Additionally, occlusion is addressed by implementing strict boundary constraints, resulting in more realistic and natural deformation outcomes. Furthermore, a stepped spatial-aware block is proposed to fuse latent multi-scale shape features in person images during depth estimation. This approach allows for creating high-precision person models in a single stage, enhancing the overall quality of the generated 3D models. Experiments conducted on the MPV-3D dataset demonstrate the superiority of the method. Regarding human modeling, Abs. decreased from 7.88 to 7.38, Sq. from 0.39 to 0.34, and RMSE from 11.27 to 10.66.