UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

Generalized softmax networks for non-linear component extraction

Lucke, J; Sahani, M; (2007) Generalized softmax networks for non-linear component extraction. In: MarquesDeSa, J and Alexandre, LA and Duch, W and Mandic, DP, (eds.) Artificial Neural Networks - ICANN 2007, Pt 1, Proceedings. (pp. 657 - 667). SPRINGER-VERLAG BERLIN

Full text not available from this repository.


We develop a probabilistic interpretation of non-linear component extraction in neural networks that activate their hidden units according to a softmaxlike mechanism. On the basis of a generative model that combines hidden causes using the max-function, we show how the extraction of input components in such networks can be interpreted as maximum likelihood parameter optimization. A simple and neurally plausible Hebbian A-rule is derived. For approximatelyoptimal learning, the activity of the hidden neural units is described by a generalized softmax function and the classical softmax is recovered for very sparse input. We use the bars benchmark test to numerically verify our analytical results and to show competitiveness of the derived learning algorithms.

Type: Proceedings paper
Title: Generalized softmax networks for non-linear component extraction
Event: 17th International Conference on Artificial Neural Networks (ICANN 2007)
Location: Oporto, PORTUGAL
Dates: 2007-09-09 - 2007-09-13
ISBN-13: 978-3-540-74689-8
URI: http://discovery.ucl.ac.uk/id/eprint/175234
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item