Lebedeva, Anna;
(2023)
Reinforcement learning across the brain.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
Thesis_corrections.pdf - Other Download (12MB) | Preview |
Abstract
During my graduate work, I was interested in investigating the activity of neural populations in different brain areas during a cognitive value-based task. In situations where no sensory stimulus is guiding animal decisions, how do they keep track of which alternative is currently the most valuable? Do they do so optimally? Which areas carry decision-related signals before the choice is made, and which ones encode the updated value signal after the outcome is revealed? What happens if activity in those areas is disrupted? This thesis has two parts. In the first part, I describe my contribution towards testing the new Neuropixels 2.0 probe to stably record from the same neurons across days, weeks and months. This technical project is the first step towards achieving the goal of tracking neural activity with high temporal resolution over time. In the second part, I describe a study of reinforcement learning across the mouse brain. Learning from rewards is a key component of cognitive flexibility in a changing world, and is often studied in humans and animals using the “dynamic two-armed bandit” task. In each trial, the subject selects one of two possible actions, each of which is associated with a different time-varying probability of reward. To maximise reward, subjects must infer which action currently has the higher reward probability and bias their choices towards that action. Although many brain regions may be involved in this task, studies to date have typically focused on a relatively small number of brain regions. This makes the comparison between regions difficult since specific tasks used across studies differ. To overcome this challenge, we developed a version of the two-armed bandit task for head-fixed mice and combined it with high-density acute extracellular recordings from ~20000 neurons across multiple brain regions, including secondary motor cortex (MOs), medial prefrontal cortex (mPFC), orbitofrontal cortex, striatum and dorsal hippocampus. We fitted and compared ten learning models of mouse behaviour in this task, and found that models that include a "habits" component matched mouse behaviour best. Among the recorded regions, MOs was distinguished by the strongest coding of the animal's 4 choice and habit. We then used optogenetics to check the causal involvement of MOs and compare it to mPFC in the task. Our findings suggest that inactivating MOs causes animals to rely less on habits, while persistent inactivation of mPFC causes slower learning rates, which can be interpreted as an increased dependence on habits. This work provides a large-scale survey of multiple brain regions, both cortical and subcortical, in the value-guided decision-making task. It highlights the involvement of MOs in habitual decision-making.
Type: | Thesis (Doctoral) |
---|---|
Qualification: | Ph.D |
Title: | Reinforcement learning across the brain |
Open access status: | An open access version is available from UCL Discovery |
Language: | English |
Additional information: | CC BY-NC: Copyright © The Author 2023. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Department of Neuromuscular Diseases |
URI: | https://discovery.ucl.ac.uk/id/eprint/10179198 |
Archive Staff Only
View Item |