UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

DetectA: abrupt concept drift detection in non-stationary environments

Escovedo, T; Koshiyama, A; da Cruz, AA; Vellasco, M; (2018) DetectA: abrupt concept drift detection in non-stationary environments. Applied Soft Computing , 62 pp. 119-133. 10.1016/j.asoc.2017.10.031. Green open access

[thumbnail of Koshiyama_AAM20170808 - Artigo Detecta - Rev to Resubmit - Final.pdf]
Preview
Text
Koshiyama_AAM20170808 - Artigo Detecta - Rev to Resubmit - Final.pdf - Accepted Version

Download (1MB) | Preview

Abstract

Almost all drift detection mechanisms designed for classification problems work reactively: after receiving the complete data set (input patterns and class labels) they apply a sequence of procedures to identify some change in the class-conditional distribution – a concept drift. However, detecting changes after its occurrence can be in some situations harmful to the process under analysis. This paper proposes a proactive approach for abrupt drift detection, called DetectA (Detect Abrupt Drift). Briefly, this method is composed of three steps: (i) label the patterns from the test set (an unlabelled data block), using an unsupervised method; (ii) compute some statistics from the train and test sets, conditioned to the given class labels for train set; and (iii) compare the training and testing statistics using a multivariate hypothesis test. Based on the results of the hypothesis tests, we attempt to detect the drift on the test set, before the real labels are obtained. A procedure for creating datasets with abrupt drift has been proposed to perform a sensitivity analysis of the DetectA model. The result of the sensitivity analysis suggests that the detector is efficient and suitable for datasets of high-dimensionality, blocks with any proportion of drifts, and datasets with class imbalance. The performance of the DetectA method, with different configurations, was also evaluated on real and artificial datasets, using an MLP as a classifier. The best results were obtained using one of the detection methods, being the proactive manner a top contender regarding improving the underlying base classifier accuracy.

Type: Article
Title: DetectA: abrupt concept drift detection in non-stationary environments
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.asoc.2017.10.031
Publisher version: https://doi.org/10.1016/j.asoc.2017.10.031
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Concept Drift; Drift Detection; Proactive Approach.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10092668
Downloads since deposit
323Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item