creators_name: Särelä, Jaakko
creators_id: 4715
editors_name: Lee, Te-Won
editors_name: Cardoso, Jean-Francois
editors_name: Oja, Erkki
editors_name: Amari, Shun-Ichi
type: journalp
datestamp: 2004-05-24
lastmod: 2011-03-11 08:55:36
metadata_visibility: show
title:  Overlearning in marginal distribution-based ICA: analysis and solutions
ispublished: pub
subjects: comp-sci-stat-model
subjects: comp-sci-mach-learn
subjects: comp-sci-neural-nets
full_text_status: public
keywords: independent component analysis, blind source separation, overlearning, overfitting, spikes, bumps, high-order ICA
abstract:   The present paper is written as a word of caution, with users of
  independent component analysis (ICA) in mind, to overlearning
  phenomena that are often observed.\\
  We consider two types of overlearning, typical to high-order
  statistics based ICA.  These algorithms can be seen to maximise the
  negentropy of the source estimates.  The first kind of overlearning
  results in the generation of spike-like signals, if there are not
  enough samples in the data or there is a considerable amount of
  noise present.  It is argued that, if the data has power spectrum
  characterised by $1/f$ curve, we face a more severe problem, which
  cannot be solved inside the strict ICA model. This overlearning is
  better characterised by bumps instead of spikes. Both overlearning
  types are demonstrated in the case of artificial signals as well as
  magnetoencephalograms (MEG). Several methods are suggested to
  circumvent both types, either by making the estimation of the ICA
  model more robust or by including further modelling of the data.

date: 2003-12
date_type: published
publication: Journal of machine learning research
volume: 4
publisher: MIT press
pagerange: 1447-1469
refereed: TRUE
citation:   Särelä, Mr Jaakko  (2003)  Overlearning in marginal distribution-based ICA: analysis and solutions.  [Journal (Paginated)]     
document_url: http://cogprints.org/3638/1/sarela03.pdf