Multi-modal data combination strategy based on chest HRCT images and PFT parameters for intelligent dyspnea identification in COPD

Because of persistent airflow limitation in chronic obstructive pulmonary disease (COPD), patients with COPD often have complications of dyspnea. However, as a leading symptom of COPD, dyspnea in COPD deserves special consideration regarding treatment in this fragile population for pre-clinical health management in COPD. Methods: Based on the above, this paper proposes a multi-modal data combination strategy by combining the local and global features for dyspnea identification in COPD based on the multi-layer perceptron (MLP) classifier. First, lung region images are automatically segmented from chest HRCT images for extracting the original 1,316 lung radiomics (OLR, 1,316) and 13,824 3D CNN features (O3C, 13,824). Second, the local features, including five selected pulmonary function test (PFT) parameters (SLF, 5), 28 selected lung radiomics (SLR, 28), and 22 selected 3D CNN features (S3C, 22), are respectively selected from the original 11 PFT parameters (OLF, 11), 1,316 OLR, and 13,824 O3C by the least absolute shrinkage and selection operator (Lasso) algorithm. Meantime, the global features, including two fused PFT parameters (FLF, 2), six fused lung radiomics (FLR, 6), and 34 fused 3D CNN features (F3C, 34), are respectively fused by 11 OLF, 1,316 OLR, and 13,824 O3C using the principal component analysis (PCA) algorithm. Finally, we combine all the local and global features (SLF + FLF + SLR + FLR + S3C + F3C, 5+ 2 + 28 + 6 + 22 + 34) for dyspnea identification in COPD based on the MLP classifier. Our proposed method comprehensively improves classification performance. The MLP classifier with all the local and global features achieves the best classification performance at 87.7% of accuracy, 87.7% of precision, 87.7% of recall, 87.7% of F1-scorel, and 89.3% of AUC, respectively. Compared with single-modal data, the proposed strategy effectively improves the classification performance for dyspnea identification in COPD, providing an objective and effective tool for COPD management.