Voice Analysis of People with Dementia:Toward to AI‐based Emotion Detection for People with Dementia

Background There is lack of evidence‐based research concerning the emotion expression of people with dementia even though it is important to detect their emotional state because emotion plays an important influence on the quality of life and communication in dementia care. Emotion detection is further important in developing AI conversation intervention programs as emotion detection could be a mediator for better Human‐machine interface. Method Researchers interview people with dementia on their emotion‐related topic (good and bad memories, their wishes, and other everyday topics) and record the dialogue. Besides, their caregivers record the dialogue with them when they show psychological and behavioral symptoms. Participants vary in gender, location of living(dialects), seriousness of the disease from mild cognitive impairment (MCI) to severe dementia. Recording is done for months long enough to collect various emotions from a person who has his or her individual voice characteristics. Researchers extract and classify the emotion into 5 categories of anger, sadness, agitation (expressed as BPSD), happiness and neutral based on the content and mood of PWD during the interview. Analysis will be done with Praat program, voice analysis program, on changes of pitch, shimmer, jitter. formant, intensity, HNR(harmony and noise ratio), and other characteristics. Verification of differences by emotions is confirmed by ANOVA, SPSS23.0. Result Statistically significant speech parameters for classifying the emotional groups are mainly related to speech characteristics such as jitter, shimmer, basic formant, intensity and HNR. For agitation and anxiety expressed by shouting, swearing, and repetition, voice features have different characteristics of intensity and HNR from ones of other emotions. Conclusion This research is useful in establishing labeling systems for future artificial intelligence modeling. At the same time, it has humanistic implications. Emotions expressed by people with dementia are various against the prejudice which their emotions are monotonous. Thus, emotion expression of people with dementia needs further analysis based on the understanding of context‐ and care method‐dependent changes. On the other hand, this data is collected in the natural settings while most data for voice‐emotion study is collected in the experimental setting. Accordingly, both limitations and merits of this unstructured data should be discussed.