UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Learning text representation using recurrent convolutional neural network with highway layers

Wen, Y; zhang, W; Luo, R; Wang, J; (2016) Learning text representation using recurrent convolutional neural network with highway layers. In: Proceedings of the SIGIR 2016 Workshop on Neural Information Retrieval. Association for Computing Machinery (ACM): Pisa, Italy. Green open access

[thumbnail of Wang_neuir2016.pdf]
Preview
Text
Wang_neuir2016.pdf

Download (262kB) | Preview

Abstract

Recently, the rapid development of word embedding and neural networks has brought new inspiration to various NLP and IR tasks. In this paper, we describe a staged hybrid model combining Recurrent Convolutional Neural Networks (RCNN) with highway layers. The highway network module is incorporated in the middle takes the output of the bidirectional Recurrent Neural Network (Bi-RNN) module in the first stage and provides the Convolutional Neural Network (CNN) module in the last stage with the input. The experiment shows that our model outperforms common neural network models (CNN, RNN, Bi-RNN) on a sentiment analysis task. Besides, the analysis of how sequence length influences the RCNN with highway layers shows that our model could learn good representation for the long text.

Type: Proceedings paper
Title: Learning text representation using recurrent convolutional neural network with highway layers
Event: Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/1235
Publisher version: https://www.microsoft.com/en-us/research/event/neu...
Language: English
Additional information: Copyright © 2016 Copyright held by the owner/author(s).
Keywords: Neural Networks, Highway Networks, Sentiment Analysis
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > UCL School of Management
URI: https://discovery.ucl.ac.uk/id/eprint/1526824
Downloads since deposit
183Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item