Rapid detection of white blood cells using hyperspectral microscopic imaging system combined with Multi-data Faster RCNN

Blood test is a basic medical test item, and the content of white blood cells (WBCs) is directly related to the evaluation of the patient's health and the judgment of serious diseases. However, both manual detection method and traditional image recognition method have the problem of insufficient efficiency and accuracy. The purpose of this study is to propose a fast and accurate WBC detection method, called hyperspectral based Multi-data Faster RCNN. For hyperspectral multi-dimensional sensing information, one-dimensional (1-D) CNN and Faster RCNN were used to model and analyze spectral and image features, respectively. And then, the Multi-data Faster RCNN was built, in which the 1-D CNN network and fusion network layer were added in Faster RCNN to realize the joint detection based on spectral and image fusion features of WBCs. The hyperspectral system coupled with Multi-data Faster RCNN can effectively improve the accuracy of WBCs’ recognition. The average precision (Ap) value and the F1 score of each classification of WBCs were above 0.9, and the mean average precision (mAP) even reached 0.962. The results showed that the Multi-data Faster RCNN integrated with hyperspectral imaging had great potential in the detection of WBCs. ●Hyperspectral multi-dimensional sensing information can be used as an effective basis for white blood cells detection.●Different kinds of white blood cells contain different enzymes which result in different spectral characteristics.●Compared with 2-D image, 1-D spectral data can achieve faster model convergence with more suitable for WBC classification.●In WBC detection, the multi-data detection has better performance in accuracy and stability than single-data detection.