Robust in-vehicle respiratory rate detection using multimodal signal fusion

Continuous health monitoring in private spaces such as the car is not yet fully exploited to detect diseases in an early stage. Therefore, we develop a redundant health monitoring sensor system and signal fusion approaches to determine the respiratory rate during driving. To recognise the breathing movements, we use a piezoelectric sensor, two accelerometers attached to the seat and the seat belt, and a camera behind the windscreen. We record data from 15 subjects during three driving scenarios (15 min each) city, highway, and countryside. An additional chest belt provides the ground truth. We compare the four convolutional neural network (CNN)-based fusion approaches: early, sensor-based late, signal-based late, and hybrid fusion. We evaluate the performance of fusing for all four signals to determine the portion of driving time and the signal combination. The hybrid algorithm fusing all four signals is most effective in detecting respiratory rates in the city ( P = 62.42 ), highway ( P = 62.67 ), and countryside ( P = 60.94 ). In summary, 60% of the total driving time can be used to measure the respiratory rate. The number of signals used in the multi-signal fusion improves reliability and enables continuous health monitoring in a driving vehicle.