Cogprints

MISEP - Linear and Nonlinear ICA Based on Mutual Information

Almeida, Luis B. (2002) MISEP - Linear and Nonlinear ICA Based on Mutual Information. [Journal (Paginated)] (Unpublished)

Full text available as:

[img]
Preview
 PDF
813Kb

Abstract

Linear Independent Components Analysis (ICA) has become an important signal processing and data analysis technique, the typical application being blind source separation in a wide range of signals, such as biomedical, acoustical and astrophysical ones. Nonlinear ICA is less developed, but has the potential to become at least as powerful. This paper presents MISEP, an ICA technique for linear and nonlinear mixtures, which is based on the minimization of the mutual information of the estimated components. MISEP is a generalization of the popular INFOMAX technique, which is extended in two ways: (1) to deal with nonlinear mixtures, and (2) to be able to adapt to the actual statistical distributions of the sources, by dynamically estimating the nonlinearities to be used at the outputs. The resulting MISEP method optimizes a network with a specialized architecture, with a single objective function: the output entropy. Examples of both linear and nonlinear ICA performed by MISEP are presented in the paper.

Item Type:Journal (Paginated)
Additional Information:Submitted to the JOurnal of Machine Learning Research
Keywords:Independent components analysis, nonlinear, blind source separation, ICA, BSS
Subjects:Computer Science > Statistical Models
Computer Science > Neural Nets
ID Code:2687
Deposited By: Almeida, Prof. Luis B.
Deposited On:05 Jan 2003
Last Modified:11 Mar 2011 08:55

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

L. B. Almeida. Multilayer perceptrons. In E. Fiesler and R. Beale, editors, Handbook of Neural Computation. Institute of Physics, Oxford University Press, 1997. available at http://www.oup-usa.org/acadref/ncc1_2.pdf .

L. B. Almeida. Linear and nonlinear ICA based on mutual information. In Proc. Symp. 2000 on Adapt. Sys. for Sig. Proc., Commun. and Control, Lake Louise, Alberta, Canada, 2000.

L. B. Almeida. Simultaneous MI-based estimation of independent components and of their distributions. In Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, pages 169-174, Helsinki, Finland, 2000.

L. B. Almeida. ICA of linear and nonlinear mixtures based on mutual information. In Proc. 2001 Int. Joint Conf. on Neural Networks, Washington, D.C., 2001.

L. B. Almeida. MISEP - an ICA method for linear and nonlinear mixtures, based on mutual information. In Proc. 2002 Int. Joint Conf. on Neural Networks, Honolulu, Hawaii, 2002.

L. B. Almeida. MISEP - linear and nonlinear ICA based on mutual information. Signal Processing, 2003. To be submitted for publication.

S. Amari, A. Cichocki, and H. H. Yang. A new learning algorithm for blind signal separation. In NIPS 95, pages 882-893. MIT Press, 1996.

A. Bell and T. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7:1129-1159, 1995.

G. Burel. Blind separation of sources: A nonlinear neural algorithm. Neural Networks, 5 (6):937-947, 1992.

J.-F. Cardoso. Infomax and maximum likelihood for source separation. IEEE Letters on Signal Processing, 4:112-114, 1997.

J.-F. Cardoso and A. Souloumiac. Jacobi angles for simultaneous diagonalization. SIAM Journal of Matrix Analysis and Applications, 17(1), 1996.

P. Comon. Independent component analysis - a new concept? Signal Processing, 36: 287-314, 1994.

G. Darmois. Analyse generale des liaisons stochastiques. Rev. Inst. Internat. Stat., 21:2-8, 1953.

G. Deco and W. Brauer. Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures. Neural Networks, 8:525-535, 1995.

S. Haykin and P. Gupta. A new activation function for blind signal separation. ASL Technical Report 1, McMaster University, Hamilton, Ontario, Canada, 1999.

A. Hyvarinen and E. Oja. A fast fixed-point algorithm for independent component analysis. Neural Computation, 9(7):1483-1492, 1997.

A. Hyvarinen and P. Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12(3):429-439, 1999.

T.-W. Lee, M. Girolami, A. Bell, and T. Sejnowski. An unifying information-theoretic framework for independent component analysis. International Journal on Mathematical and Computer Modeling, 1998.

T.-W. Lee, M. Girolami, and T. Sejnowski. Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources. Neural Computation, 11:417-441, 1999.

G. C. Marques and L. B. Almeida. An objective function for independence. In Proc. International Conference on Neural Networks, pages 453-457, Washington DC, 1996.

G. C. Marques and L. B. Almeida. Separation of nonlinear mixtures using pattern repulsion. In J. F. Cardoso, C. Jutten, and P. Loubaton, editors, Proc. First Int. Worksh. Independent Component Analysis and Signal Separation, pages 277-282, Aussois, France, 1999.

A. Taleb and C. Jutten. Entropy optimization - application to blind separation of sources. In Proc. ICANN'97, Lausanne, Switzerland, 1997.

H. Valpola. Nonlinear independent component analysis using ensemble learning: Theory. In Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, pages 251-256, Helsinki, Finland, 2000.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.