UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Improving a statistical language model through non-linear prediction

Mnih, A; Zhang, YC; Hinton, G; (2009) Improving a statistical language model through non-linear prediction. In: NEUROCOMPUTING. (pp. 1414 - 1418). ELSEVIER SCIENCE BV

Full text not available from this repository.


We show how to improve a state-of-the-art neural network language model that converts the previous "context" words into feature vectors and combines these feature vectors linearly to predict the feature vector of the next word. Significant improvements in predictive accuracy are achieved by using a non-linear subnetwork to modulate the effects of the context words or to produce a non-linear correction term when predicting the feature vector. A log-bilinear language model that incorporates both of these improvements achieves a 26% reduction in perplexity over the best n-gram model on a fairly large dataset. (C) 2009 Elsevier B.V. All rights reserved.

Type: Proceedings paper
Title: Improving a statistical language model through non-linear prediction
Event: 18th European Symposium on Artificial Neural Networks
Location: Brugge, BELGIUM
Dates: 2008-04
DOI: 10.1016/j.neucom.2008.12.025
Keywords: Statistical language modelling, Distributed representations, Neural networks
URI: http://discovery.ucl.ac.uk/id/eprint/1304517
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item