What can the Body Tell Us About the Brain: Machine Learning for Automation

Our ability to record large-scale neural and behavioral data has substantially improved in the last decade. However, the inference of quantitative dynamical models for cognition and motor control remains challenging due to their unconstrained nature. Here, we incorporate constraints from anatomy and physiology to tame machine learning models of neural activity and behavior. I will show that these constraints-based modeling approaches allow us to predictively understand the relationship between neural activity and behavior. How does the motor cortex achieve generalizable and purposeful movements from the complex, nonlinear musculoskeletal system? I will present a deep reinforcement learning framework for training recurrent neural network controllers that act on anatomically accurate limb models such that they achieve desired movements. We apply this framework to kinematic and neural recordings made in macaques as they perform movements at different speeds. This framework for the control of the musculoskeletal system mimics biologically observed neural strategies and enables hypothesis generation for prediction and analysis of novel movements and neural strategies.

Modeling neural activity and behavior across different subjects and in a naturalistic setting remains a significant challenge. Here, we develop novel explainable AI methods for modeling continuously varying differences in behavior, which successfully represent distinct features of multi-subject and social behavior in an unsupervised manner. These methods are also successful at uncovering the relationships between recorded neural data and the ensuing behavior. We end with future work on understanding the interactions between two or more subjects during social behavior.

Learning Objectives:

1. Recognize the role of behavior in constrained machine learning models of the brain.

2. Discuss the different levels at which we can model behavior.

3. Summarize the downstream applications on motor control, modeling inter-subject variability, and social behavior.