The roles and interplay between reinforcement-based and error-based processes on sensorimotor exploration
Date
2024
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
From a baby's babbling to relearning functional motor skills following a neurological injury, exploration has been widely seen as critical to success. Recently, movement variability has been posited as a potential mechanism to promote sensorimotor exploration [1, 2, 3, 4, 5]. Yet it remains unclear what are the fundamental mechanisms that govern sensorimotor exploration through movement variability. An open question is whether utilizing movement variability to explore motor tasks is conditioned on reinforcement feedback (whether an action was successful), error feedback (dependent on magnitude and direction of error), or their interplay. The objective of this dissertation is gain insight into how movement variability can be utilized to explore motor tasks. This goal was accomplished through a complementary blend of behavioral experiments and computational modeling. ☐ In Aim 1, we established that reinforcement-based processes actively regulate exploratory behavior along redundant solution manifolds. Across three reaching experiments we manipulated the size of both the visually displayed target and an unseen reward zone as well as the probability of reinforcement feedback. Our empirical and modeling results support the idea that exploratory random walk behavior emerges by utilizing knowledge of movement variability to update intended reach aim towards recently reinforced motor actions. ☐ In Aim 2, we investigate how different modes of reinforcement--positive reward and positive punishment--influence movement variability and sensorimotor adaptation. We recruited over 100 participants across a sensorimotor adaptation task and a task that does not require adaptation to be successful. Contrary to previous findings, we found that punishment feedback leads to a greater extent of learning, but reduced movement variability, compared to reward feedback. ☐ In Aim 3, we investigate the roles and interplay between reinforcement-based and error-based on sensorimotor exploration. Across two reaching experiments and a re-analysis of previous work in gait, we show evidence to suggest that reinforcement-based processes boost exploration while error-based processes suppress exploration. When acting in concert, reinforcement-based and error-based processes mutually oppose one another to result in moderate levels of exploration. Additionally, we gain causal insight into the role of reinforcement-based neural processes on sensorimotor exploration by recruiting participants with Parkinson's disease, a disease known to compromise reinforcement-base neural circuitry. ☐ This work is important because we have very limited information on the independent contribution of and interaction between reinforcement-based and error-based processes during motor exploration. The knowledge gained here may pave the way to develop better informed and effective neurorehabilitation for those recovering from a neurological disease or injury.
Description
Keywords
Error-based processes, Movement variability, Reinforcement feedback, Sensorimotor neuroscience