A unifying information-theoretic framework for independent component analysis.
COMPUT MATH APPL
We show that different theories recently proposed for independent component analysis (ICA) lead to the same iterative learning algorithm for blind separation of mixed independent sources. We review those theories and suggest that information theory can be used to unify several lines of research. Pearlmutter and Parra  and Cardoso  showed that the infomax approach of Bell and Sejnowski  and the maximum likelihood estimation approach are equivalent. We show that negentropy maximization also has equivalent properties, and therefore, all three approaches yield the same learning rule for a fixed nonlinearity. Girolami and Fyfe  have shown that the nonlinear principal component analysis (PCA) algorithm of Karhunen and Joutsensalo  and Oja  can also be viewed from information-theoretic principles since it minimizes the sum of squares of the fourth-order marginal cumulants, and therefore, approximately minimizes the mutual information . Lambert  has proposed different Bussgang cost functions for multichannel blind deconvolution. We show how the Bussgang property relates to the infomax principle. Finally, we discuss convergence and stability as well as future research issues in blind source separation. (C) 2000 Elsevier Science Ltd. All rights reserved.
|Title:||A unifying information-theoretic framework for independent component analysis|
|Keywords:||blind source separation, ICA, entropy, information maximization, maximum likelihood estimation, BLIND SIGNAL SEPARATION, LEARNING ALGORITHMS, PROJECTION PURSUIT, MUTUAL INFORMATION, MIXTURE, REPRESENTATION, DISTRIBUTIONS, MAXIMIZATION, NETWORK, RULE|
|UCL classification:||UCL > School of BEAMS > Faculty of Maths and Physical Sciences
UCL > School of BEAMS > Faculty of Maths and Physical Sciences > Statistical Science
Archive Staff Only