Sharkey, Edward; (2025) Transformers for Commodity Forecasting. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Edward_Sharkey_Thesis_Submission.pdf - Accepted Version Download (22MB) | Preview

Abstract

This thesis investigates the application of Transformer models for commodity forecasting (e.g., Copper), using Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformers (GPT) model architectures. The research has five dimensions: firstly, a comparison of BERT and GPT; secondly, we built fine-tuned copper models (CopBERT, CopGPT); thirdly, we built a method for optimising Large Language Model (LLM) prompts (AgentOptimiser); fourthly, we built a signal detection model to identify price change signals within the metal futures market; finally, we built several mechanisms for mechanistic interpretability in our commodity models attempting to decipher the internal workings. The research highlights the significance of mechanistic interpretability and metaheuristics for prompt optimisation to enhance accuracy and interpretability. Mechanistic interpretability refers to understanding how a machine learning model or algorithm functions at a detailed, structural level. Metaheuristics refers to a broad class of optimisation techniques that are used to find near-optimal solutions to complex optimisation problems, particularly when the problem space is large and complex. Practical testing and trials of the models were conducted with a metal trading desk; feedback was obtained from energy companies and several large global telecommunication companies.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item