Cousins, Simon; (2019) Discriminative learning for structured outputs and environments. Doctoral thesis (Ph.D), UCL (University College London).

Preview

Text Cousins_Thesis.pdf Download (1MB) | Preview

Abstract

Machine learning methods have had considerable success across a wide range of applications. Much of this success is due to the flexibility of learning algorithms and their ability to tailor themselves to the requirements of the particular problem. In this thesis we examine methods that seek to exploit the underlying structure of a problem and make the best possible use of the available data. We explore the structural nature of two different problems, binary classification under the uncertainty of input relationships, and multi-label output learning of Markov networks with unknown graph structures. From the input perspective, we focus on binary classification and the problems associated with learning from limited amounts of data. In particular we pay attention to moment based methods and investigate how to deal with the uncertainty surrounding the estimate of moments using either small or noisy training samples. We present a worst-case analysis and show how the high probability bounds on the deviation of the true moments from their empirical counterparts can be used to generate a regularisation scheme that takes into consideration the relative amount of information that is available for each class. This results in a binary classification algorithm that directly minimises the worst case future misclassification rate, whilst taking into consideration the possible errors in the moment estimates. This algorithm was shown to outperform a number of traditional approaches across a range of benchmark datasets, doing particularly well when training was limited to small amounts of data. This supports the idea that we can leverage the class specific regularisation scheme and take advantage of the uncertainty of the datasets when creating a predictor. Further encouragement for this approach was provided during the high-noise experiments, predicting the directional movement of popular currency pairs, where moment based methods outperformed those using the peripheral point of the class-conditional distributions. From the output perspective, we focus on the problem of multi-label output learning over Markov networks and present a novel large margin learning method that leverages the correlation between output labels. Our approach is agnostic to the output graph structure and it simultaneously learns the intrinsic structure of the outputs, whilst finding a large margin separator. Based upon the observation that the score function over the complete output graph is given by the expectation of the score function over all spanning trees, we formulate the problem as an L1-norm multiple kernel learning problem where each spanning tree over the complete output graph gives rise to a particular instance of a kernel. We show that this approach is comparable to state-of-the-art approaches on a number of benchmark multi-label learning problems. Furthermore, we show how this method can be applied to the problem of predicting the joint movement of a group of stocks, where we not only infer the directional movement of individual stocks but also uncover insights on the input-dependent relationships between them.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item