UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

LOBO: Evaluation of Generalization Deficiencies in Twitter Bot Classifiers

Echeverria Guzman, J; De Cristofaro, E; Kourtellis, N; Leontiadis, I; Stringhini, G; Zhou, S; (2018) LOBO: Evaluation of Generalization Deficiencies in Twitter Bot Classifiers. In: ACSAC '18 Proceedings of the 34th Annual Computer Security Applications Conference. (pp. pp. 137-146). ACM Green open access

[img]
Preview
Text
arxiv.pdf - ["content_typename_Published version" not defined]

Download (332kB) | Preview

Abstract

Botnets in online social networks are increasingly often affecting the regular flow of discussion, attacking regular users and their posts, spamming them with irrelevant or offensive content, and even manipulating the popularity of messages and accounts. Researchers and cybercriminals are involved in an arms race, and new and updated botnets designed to defeat current detection systems are constantly developed, rendering such detection systems obsolete. In this paper, we motivate the need for a generalized evaluation in Twitter bot detection and propose a methodology to evaluate bot classifiers by testing them on unseen bot classes. We show that this methodology is empirically robust, using bot classes of varying sizes and characteristics and reaching similar results, and argue that methods trained and tested on single bot classes or datasets might not able to generalize to new bot classes. We train one such classifier on over 200,000 data points and show that it achieves over 97% accuracy. The data used to train and test this classifier includes some of the largest and most varied collections of bots used in literature. We then test this theoretically sound classifier using our methodology, highlighting that it does not generalize well to unseen bot classes. Finally, we discuss the implications of our results, and reasons why some bot classes are easier and faster to detect than others.

Type: Proceedings paper
Title: LOBO: Evaluation of Generalization Deficiencies in Twitter Bot Classifiers
Event: 34th Annual Computer Security Applications Conference, 3 - 7 December 2018, San Juan, Puerto Rico
Location: San Juan, Puerto Rico
Dates: 03 December 2018 - 07 December 2018
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3274694.3274738
Publisher version: https://doi.org/10.1145/3274694.3274738
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: http://discovery.ucl.ac.uk/id/eprint/10057050
Downloads since deposit
9Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item