Carrasco-Davis, Rodrigo; (2025) Principles of Optimal Learning Control in Biological and Artificial Agents. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text thesis_final.pdf - Accepted Version Download (17MB) | Preview

Abstract

Humans and other animals must continuously decide how to allocate their cognitive resources. A key aspect of this involves controlling effort throughout the learning of new tasks, known as cognitive control of learning. In machine learning, this is referred to as meta-learning and focuses on scheduling hyperparameters to improve the learning processes. Determining the optimal allocation of control requires complex computations, such as estimating future utility while accounting for uncertainties, learning capacity, task difficulty, and other resource demands. Although previous research has proposed computational mechanisms for control allocation in learning, these methods often lack interpretability, resist mathematical analysis, and rely on models that are difficult to scale. This thesis adopts a parsimonious approach to identify principles of learning control. It introduces a normative framework based on cumulative reward maximization for optimally allocating control throughout the learning process. This framework unifies and instantiates existing theories in machine learning, such as Model-Agnostic Meta-Learning, and in cognitive neuroscience, specifically the Expected Value of Control theory applied to learning systems. The thesis further explores the application of this framework to neural networks, revealing key features of optimal learning control, including inter-temporal control allocation and curriculum learning. Additionally, it provides mathematical analyses for optimal learning control, approximating time-dependent control allocation using methods from control theory. Another aspect of meta-learning is identifying learnable task components. To achieve this, the thesis introduces a novel method for estimating epistemic uncertainty to prioritize replay on the most useful experiences. The primary contribution of this thesis is a formal normative framework for understanding learning control in machine learning and cognitive neuroscience. It provides methods for optimal control and epistemic uncertainty computation, suggesting directions for further mathematical analysis. This work could establish a robust framework for understanding animal learning across lifespans and enable more efficient learning in artificial systems.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as CSVJSONXML

Archive Staff Only

View Item