An End-to-End framework for extracting observable cues of depression from diary recordings

Because of the prevalence of depression, its often-chronic course, relapse and associated disability, early detection and non-intrusive monitoring is a crucial tool for timely diagnosis and treatment, remission of depression and prevention of relapse. In this way, its impact on quality of life and well-being can be limited. Current attempts to use artificial intelligence for the early classification of depression are mostly data-driven and thus non-transparent and lack effective means to deal with uncertainties. Therefore, in this paper, we propose an end-to-end framework for extracting observable depression cues from diary recordings. Furthermore, we also explore its feasibility for automatic detection of depression symptoms using observable behavioural cues. The proposed end-to-end framework for extracting depression was used to evaluate 28 video recordings from the Symptom Media dataset and 27 recordings from the DAIC-WOZ dataset. We compared the presence of the extracted features between recordings of individuals with and without a depressive disorder. We identified several cues consistent with previous studies in terms of their differentiation between individuals with and without depressive disorder across both datasets among language (i.e., use of negatively valanced words, use of first-person singular pronouns, some features of language complexity, explicit mentions of treatment for depression), speech (i.e., monotonous speech, voiced speech and pauses, speaking rate, low articulation rate), and facial cues (i.e., rotational energy of head movements). The nature/context of the discourse, the impact of other disorders and physical/psychological stress, and the quality and resolution of the recordings all play an important role in matching the digital features to the relevant background. In this way, the work presented in this paper provides a novel approach to extracting a wide range of cues relevant to the classification of depression and opens up new opportunities for further research.