Greenall, William;
(2024)
Orthogonality in Machine Learning.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
PhdThesisWilliamGreenallFinal19085084.pdf - Accepted Version Download (6MB) | Preview |
Abstract
In this thesis I focus on the applications and relevance of orthogonality in various topics in machine learning. The theme of the thesis is that different viewpoints of the concept of orthogonality and Hilbert spaces in general can be utilised to improve the performance of machine learning algorithms, as well as inform development of new ones. The approach taken focuses in part on the rich and interesting theory of orthogonal polynomials, which are heretofore underutilised in machine learning methods as a tool for feature construction \par First, I look at a sparse Gaussian process schema relying on appropriate construction of orthogonal basis functions, as well as relevant theory that shows that orthonormality is an important feature of the chosen sparse method. This yields a novel approach to feature construction and sparse Gaussian process regression. \par Next, I utilise orthogonality and an appropriately defined inner product as a tool for a new form of interpretable feature construction in problems with dynamic graphs. The approach centres on comparison between graphs via an implicit measure of orthogonality of their matching polynomials. This is applied to anomaly detection as a guiding example, using a "landmarks" strategy. \par Finally, I propose a new type of Gaussian Cox process, which yields application of orthogonal series estimate models in order to construct a rapid Bayesian inference scheme, bypassing the usual difficulties of the highly non-Gaussian likelihood. This is then extended, through appropriate approximation schemata for higher-order Gaussian moments, to stochastic classification models, yielding a rapid and flexible stochastic classifier, whose predictions can be interpreted as exact probabilities and yield direct uncertainty quantification. This stands in contrast to standard models that train on degenerate distributions to yield probabilistic predictions in an ad-hoc fashion.
| Type: | Thesis (Doctoral) |
|---|---|
| Qualification: | Ph.D |
| Title: | Orthogonality in Machine Learning |
| Open access status: | An open access version is available from UCL Discovery |
| Language: | English |
| Additional information: | Copyright © The Author 2024. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10193735 |
Archive Staff Only
 |
View Item |
