UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Generalisation in Named Entity Recognition: A Quantitative Analysis

Augenstein, I; Derczynski, L; Bontcheva, K; (2017) Generalisation in Named Entity Recognition: A Quantitative Analysis. Computer Speech & Language , 44 pp. 61-83. 10.1016/j.csl.2017.01.012. Green open access

[thumbnail of 1-s2.0-S088523081630002X-main.pdf]
Preview
Text
1-s2.0-S088523081630002X-main.pdf - Published Version

Download (602kB) | Preview

Abstract

Named Entity Recognition (NER) is a key NLP task, which is all the more challenging on Web and user-generated content with their diverse and continuously changing language. This paper aims to quantify how this diversity impacts state-of-the-art NER methods, by measuring named entity (NE) and context variability, feature sparsity, and their effects on precision and recall. In particular, our findings indicate that NER approaches struggle to generalise in diverse genres with limited training data. Unseen NEs, in particular, play an important role, which have a higher incidence in diverse genres such as social media than in more regular genres such as newswire. Coupled with a higher incidence of unseen features more generally and the lack of large training corpora, this leads to significantly lower F1 scores for diverse genres as compared to more regular ones. We also find that leading systems rely heavily on surface forms found in training data, having problems generalising beyond these, and offer explanations for this observation.

Type: Article
Title: Generalisation in Named Entity Recognition: A Quantitative Analysis
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.csl.2017.01.012
Publisher version: http://doi.org/10.1016/j.csl.2017.01.012
Language: English
Additional information: Copyright © 2017 The Authors. Published by Elsevier Ltd. This is an open access article published under the CC BY license(http://creativecommons.org/licenses/by/4.0/).
Keywords: Natural language processing; Information extraction; Named entity recognition; Generalisation; Entity drift; Social media; Quantitative study
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1547708
Downloads since deposit
179Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item