OBM-CNN: a new double-stream convolutional neural network for shield pattern segmentation in ancient oracle bones

The rejoining of oracle bone (OB) rubbings is a fundamental topic in oracle research. However, severely damaged OB fragments usually lack important material information for rejoining, which has limited the progress of OB rejoining research. Identifying material information from broken OBs is difficult because interclass differences are so obscure. In this case, we can only make a judgment by relying on the “shield pattern” presented in OB rubbings. However, it is time-consuming and laborious to identify materials from “shield pattern” identification directly, and the classification accuracy is typically very low. Thus, we proposed a novel two-stream convolutional neural network (OBM-CNN), which consists of segmentation and detection subnetworks, to handle this challenge. First, the segmentation subnetwork is based on UNet++ and improved with residual block bilinear interpolation. Then, in the detection subnetwork, the backbone feature extraction network of Faster RCNN is replaced with the encoder feature extraction network, and the detection accuracy is significantly improved through cross-training. In addition, a novel dataset named OB-Material was constructed, and we provided labels for the segmentation and detection of “shield patterns”, which compensated for the lack of oracle material datasets. The experimental results show that the “shield pattern” segmentation of our proposed method reached an F1-score value of 95.23%. For “shield pattern” detection, when IoU= 0.5, the F-score value was 9.72% higher than that of the optimal contrast model. In material classification, the optimal accuracy rate reached an excellent result of 91.8%. In conclusion, this paper presents value data for promoting the combination of oracle bone inscriptions and AI technology and the application of AI-aided ancient Chinese characters.