creators_name: Almeida, Luis B. type: confpaper datestamp: 2003-03-30 lastmod: 2011-03-11 08:55:14 metadata_visibility: show title: Faster Training in Nonlinear ICA using MISEP ispublished: pub subjects: comp-sci-stat-model subjects: comp-sci-mach-learn subjects: comp-sci-neural-nets full_text_status: public keywords: Independent components analysis, nonlinear, blind source separation, ICA, BSS abstract: MISEP has been proposed as a generalization of the INFOMAX method in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs, making the method suitable to the separation of components with a wide range of statistical distributions. In all implementations up to now, MISEP had used multilayer perceptrons (MLPs) to perform the nonlinear ICA operation. Use of MLPs sometimes leads to a relatively slow training. This has been attributed, at least in part, to the non-local character of the MLP's units. This paper investigates the possibility of using a network of radial basis function (RBF) units for performing the nonlinear ICA operation. It shows that the local character of the RBF network's units allows a significant speedup in the training of the system. The paper gives a brief introduction to the basics of the MISEP method, and presents experimental results showing the speed advantage of using an RBF-based network to perform the ICA operation. date: 2003 date_type: published refereed: TRUE referencetext: G. Burel, ``Blind separation of sources: A nonlinear neural algorithm,'' Neural Networks, vol. 5, no. 6, pp. 937--947, 1992. G. Deco and W. Brauer, ``Nonlinear higher­order statistical decorrelation by volume­conserving neural architectures,'' Neural Networks, vol. 8, pp. 525--535, 1995. G. C. Marques and L. B. Almeida, ``An objective function for independence,'' in Proc. International Conference on Neural Networks, Washington DC, 1996, pp. 453--457. G. C. Marques and L. B. Almeida, ``Separation of nonlinear mixtures using pattern repulsion,'' in Proc. First Int. Worksh. Independent Component Analysis and Signal Separation, J. F. Cardoso, C. Jutten, and P. Loubaton, Eds., Aussois, France, 1999, pp. 277--282. H. Valpola, ``Nonlinear independent component analysis using ensemble learning: Theory,'' in Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, Helsinki, Finland, 2000, pp. 251--256. L. B. Almeida, ``Linear and nonlinear ICA based on mutual information,'' in Proc. Symp. 2000 on Adapt. Sys. for Sig. Proc., Commun. and Control, Lake Louise, Alberta, Canada, 2000. L. B. Almeida, ``Simultaneous MI­based estimation of independent components and of their distributions,'' in Proc. Second Int. Worksh. Independent Component Analysis and Blind Signal Separation, Helsinki, Finland, 2000, pp. 169--174. L. B. Almeida, ``MISEP ­ linear and nonlinear ICA based on mutual information,'' Journal of Machine Learning Research, submitted for publication; available at http://neural.inesc­id.pt/lba/papers/jmlr03.pdf. A. Bell and T. Sejnowski, ``An information­maximization approach to blind separation and blind deconvolution,'' Neural Computation, vol. 7, pp. 1129--1159, 1995. L. B. Almeida, ``Multilayer perceptrons,'' in Handbook of Neural Computation, E. Fiesler and R. Beale, Eds. Institute of Physics, 1997, Oxford University Press, available at http://www.iop.org/Books/CIL/HNC/pdf/NCC1\_2.PDF. J. Moody and C. Darken, ``Learning with localized receptive fields,'' in Proc. 1988 Connectionist Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. 1988, pp. 133--143, Morgan Kaufmann, San Mateo, CA. citation: Almeida, Luis B. (2003) Faster Training in Nonlinear ICA using MISEP. [Conference Paper] document_url: http://cogprints.org/2854/1/AlmeidaICA2003.pdf