Estimation of urban-scale photovoltaic potential: A deep learning-based approach for constructing three-dimensional building models from optical remote sensing imagery

•A deep learning based approach for constructing 3D buildings from satellite imagery was developed.•Rooftop segmentation and building height prediction are satisfactory.•Estimated PV potentials derived from the actual and predicted buildings showed little difference.•Proposed approach can facilitate PV penetration and urban studies in various fields. Building-integrated photovoltaics are increasingly used to build low-carbon buildings and promote energy transition. However, the absence of three-dimensional (3D) building models may hinder accurate estimation of photovoltaic (PV) potential on 3D urban surfaces. This study develops a detail-oriented deep learning approach, which for the first time constructs 3D buildings from high-resolution satellite images and estimates PV potential. Specifically, two convolutional neural networks, i.e., the Rooftop Segmentation Model and Height Prediction Model, were developed by advancing the basic DeepLabv3+ architecture and integrating dedicated layers, adaptive activation functions, and hybrid losses. Next, the two models were trained and tested on a self-made dataset targeted at Shanghai and an open datasets under standard data augmentation and transfer learning strategies. Then, morphological post-processing procedures were developed to cluster and regularize individual rooftops with estimated heights. Finally, PV potentials in typical areas were estimated and compared. Accuracy assessments suggest satisfactory rooftop segmentationand building height estimation. The absolute relative error between the PV potentials derived from the actual and predicted building models showed little difference, implying the reliability of the extracted buildings. The proposed model is novel and effective for constructing 3D building models that can facilitate PV penetration and urban studies in various fields.