UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning

He, Z; Gao, S; Xiao, L; Barber, D; (2017) Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning. In: Guyon, I and Luxburg, UV and Bengio, S and Wallach, H and Fergus, R and Vishwanathan, S and Garnett, R, (eds.) Advances in Neural Information Processing Systems 30 (NIPS 2017) pre-proceedings. NIPS Proceedings: Long Beach, CA, USA. Green open access

[img]
Preview
Text
Barber_6606-wider-and-deeper-cheaper-and-faster-tensorized-lstms-for-sequence-learning.pdf - Published version

Download (3MB) | Preview

Abstract

Long Short-Term Memory (LSTM) is a popular approach to boosting the ability of Recurrent Neural Networks to store longer term temporal information. The capacity of an LSTM network can be increased by widening and adding layers. However, usually the former introduces additional parameters, while the latter increases the runtime. As an alternative we propose the Tensorized LSTM in which the hidden states are represented by tensors and updated via a cross-layer convolution. By increasing the tensor size, the network can be widened efficiently without additional parameters since the parameters are shared across different locations in the tensor; by delaying the output, the network can be deepened implicitly with little additional runtime since deep computations for each timestep are merged into temporal computations of the sequence. Experiments conducted on five challenging sequence learning tasks show the potential of the proposed model.

Type: Proceedings paper
Title: Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning
Event: Neural Information Processing Systems 2017
Location: Long Beach, CA, USA
Dates: 04 December 2017 - 09 December 2017
Open access status: An open access version is available from UCL Discovery
Publisher version: https://papers.nips.cc/paper/6606-wider-and-deeper...
Language: English
Additional information: This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10038401
Downloads since deposit
69Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item