An Emerging Multimodal Ecosystem for Psychopathology Research

The study of psychopathology is changing with the advent and pervasive use of smartphones, wearables, and connected devices together with significant advances in brain imaging. These new avenues offer a unique opportunity to understand phenotypic variation in a large population, to track individuals longitudinally, and relate these data to life and clinical outcomes. Despite modern sensors’ ability to capture real-world behavioral data at high temporal frequency and over long periods of time, access to and availability of longitudinal data about individuals is severely lacking. In this talk, a few of these approaches connecting brain imaging, voice, and language to psychopathology are presented through the lens of analytical approaches spanning deep learning, signal processing, and natural language processing. The integration of these sensors and newer analytic techniques will enable a change from the current symptomatic and subjective assessment and treatment of mental health to direct quantitative measurement of behavior and brain function and subsequent adjustments to alter mental health.

Learning objectives:

1. Summarize how different data modalities provide different views of brain disorders.

2. Review the use of ML/AI tools in psychopathology research.

3. Explain the role of voice, speech and language as a window into the mind.