UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Adaptive regularization for Lasso models in the context of non-stationary data streams

Monti, RP; Anagnostopoulos, C; Montana, G; (2018) Adaptive regularization for Lasso models in the context of non-stationary data streams. Statistical Analysis and Data Mining , 11 (5) , Article 11390. 10.1002/sam.11390. Green open access

[thumbnail of SADM_final.pdf]
Preview
Text
SADM_final.pdf - Accepted Version

Download (997kB) | Preview

Abstract

Large‐scale, streaming data sets are ubiquitous in modern machine learning. Streaming algorithms must be scalable, amenable to incremental training, and robust to the presence of non-stationarity. In this work we consider the problem of learning ℓ1 regularized linear models in the context of streaming data. In particular, the focus of this work revolves around how to select the regularization parameter when data arrives sequentially and the underlying distribution is nonstationary (implying the choice of optimal regularization parameter is itself time‐varying). We propose a framework through which to infer an adaptive regularization parameter. Our approach employs an ℓ1 penalty constraint where the corresponding sparsity parameter is iteratively updated via stochastic gradient descent. This serves to reformulate the choice of regularization parameter in a principled framework for online learning. The proposed method is derived for linear regression and subsequently extended to generalized linear models. We validate our approach using simulated and real data sets, concluding with an application to a neuroimaging data set.

Type: Article
Title: Adaptive regularization for Lasso models in the context of non-stationary data streams
Open access status: An open access version is available from UCL Discovery
DOI: 10.1002/sam.11390
Publisher version: https://doi.org/10.1002/sam.11390
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: ℓ1 regularization, adaptive filtering, time-varying sparsity, non-stationary data streams
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit
URI: https://discovery.ucl.ac.uk/id/eprint/10057547
Downloads since deposit
60Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item