Facial- and speech- behavioural markers associated with depression

Abstract

Background: Depression is amongst one of the most prevalent and debilitating mental illnesses, making effective detection and assessment of depression of paramount importance. While traditional methods of assessment require resources in terms of human expertise, time and finances, behavioural markers that are externally observable and objectively measurable emerged at a cost effective and efficient alternative in depression detection and assessment. In addition, behavioural markers may potentially address the inherent limitations in traditional assessment measures, namely reporting bias. Prior studies indicate that certain facial- and speech- behavioural markers are associated with depressed individuals. However, existing studies supportive of using facial and speech behavioural markers as measures for depression are predominantly based in Caucasian population, while little studies have been conducted directly on Asian population. Hence, current study aims to validate the association between facial and speech behavioural markers within the Asian population. Methods: Facial and speech features of 95 participant recruited from the University of Hong Kong were extracted using a mobile phone application, with three emotion conditions (Neutral, Happy, Sad) induced via autobiographical recalls. During autobiographical recall tasks, facial features measured by action units (AU) that represent movements of individual facial muscles were extracted; while speech features including prosodic, source and spectral features from speech production were also extracted. After extraction of these behavioural markers, Pearson’s correlation analyses were used to examine the bivariate association between respective behavioural marker and depression severity, with adjustment using the Benjamini-Hochberg procedure (FDR); then linear regressions analyses were used to examine the independent predictive power of significant behavioural markers within and across the two multimodality. Results: Without Benjamini-Hochberg adjustment, Pearson’s correlation analyses indicate that in terms of facial features, movements of AU1 (Inner Brow Raiser), AU4 (Brow Lowerer), and AU15 (Lip Corner Depressor) were positively correlated with depression severity across different emotion condition, consistent with existing findings among Caucasian population. In terms of speech features, pitch, shimmer, and jitter were negatively correlated with depression severity across different emotion condition, showing some consistency with findings among Caucasian population. Among facial features, AU4 (Brow Lowerer) remained an independent predictor of depression severity. Whereas, no single speech feature was identified as an independent predictor, when all significant speech features were entered simultaneously to predict depression severity. However, it is worth noting that after adjustment with Benjamini-Hochberg procedure, none of the behavioural markers retained the correlational significance. Limitation: The study was conducted in non-standardised setting (not in laboratory setting), which possibly compromises on the quality and consistency of data collected. Separately, it is limited by the language discrepancy where majority of participants recruited were native Chinese speakers, while the autobiographical recall tasks were conducted in English. Conclusion: Findings only provided weak evidence that certain facial (AU1- Inner Brow Raiser, AU4- Brow Lowerer, and AU15- Lip Corner Depressor) and speech (pitch, shimmer, and jitter) behavioural markers might be associated with depression severity among Hong Kong population. However, further study conducted in standardised laboratory setting is recommended to validate and generalise these findings.