Vision-based volumetric measurements via deep learning-based point cloud segmentation for material management in jobsites

Emerging vision-based frameworks have demonstrated the great potential to robustly perform volumetric measurements on point cloud models, which has several applications for site material management (e.g., during earthworks). However, prevalent vision-based frameworks to date involve human interventions to manually trim objects of interest from point cloud models, which would be time-consuming and labor-intensive. In addition, point cloud models for volumetric measurements are often incomplete and noisy. To address such challenges, we automatically detect and segment target objects in point cloud models via a deep learning-based approach and then map the semantic values onto point cloud models for 3D semantic segmentation. Once target objects are segmented, the associated volumes are quantified through the proposed vision-based computational process. For evaluation, case studies were performed on material piles in the real-world. The proposed method has the potential to enhance vision-based volumetric measurements, which supports systematic decision-making for material management in jobsites. •New vision-based volumetric measurements to reduce human intervention.•3D point cloud segmentation based on 2D semantic segmentation.•The integration of volumetric measurements with vision-based object recognition.•Results show the potential for enhanced vision-based volumetric measurements.