Towards real-world wearable sleepiness detection: Electrodermal activity data during speech can identify sleep deprivation

Accurate assessment of sleepiness is pivotal in managing the fatigue-associated risks stemming from sleep deprivation. Speech signals are easy to obtain, allowing detection of sleepiness anywhere. Previous machine learning (ML) studies using speech have not been successful in achieving reliable estimation of perceived sleepiness levels, which results in inaccurate sleepiness determination. In this paper, we propose that these challenges primarily stem from the inherent complexities of speech signals with inaccurate labels of sleepiness. Because the physical effects of sleepiness become pronounced after prolonged wakefulness, we conducted a 25-h sleep deprivation study. We collected electrodermal activity (EDA) and speech data from 30 subjects during speech production every 2 h over the 25-hour period, along with various sleepiness level labels—their cognitive impairment scores derived from the psychomotor vigilance test, their self-reported sleepiness scores, and the h awake scores. The data analysis compared EDA recorded during speech versus only the speech data and examined which approach provided better sleepiness level estimation and detection using ML. The ML result is that features derived from only EDA during speech production provided the most accurate sleepiness determination. Specifically, EDA ML models trained using the hours awake scores provided the best sleepiness level estimation, with 0.53 correlation, and better detection of sleepiness (which is related to cognitive performance deterioration), with 0.85 accuracy (0.80 sensitivity), when compared to ML features derived from speech, which obtained 0.40 correlation for sleepiness level estimation and 0.69 accuracy (0.59 sensitivity) for sleepiness detection. Moreover, the EDA data collected during speech production offered the best performance for sleepiness detection compared to EDA collected during other activities, such as visual vigilance (0.68 accuracy and 0.65 sensitivity). Given the potential of EDA data during speech production, this work demonstrates the promise of future wearable devices that could collect EDA data from speech activity, along with speech signals, for more advanced and accurate real-world sleepiness detection. [Display omitted] •Speech conveys diverse information through vocal components, while electrodermal activity (EDA) offers a more direct measure of sympathetic nervous system (SNS) activity.•Sleep deprivation significantly involves physical fatigue, which induce