---
abstract: |2
    The present paper is written as a word of caution, with users of
    independent component analysis (ICA) in mind, to overlearning
    phenomena that are often observed.\\
    We consider two types of overlearning, typical to high-order
    statistics based ICA.  These algorithms can be seen to maximise the
    negentropy of the source estimates.  The first kind of overlearning
    results in the generation of spike-like signals, if there are not
    enough samples in the data or there is a considerable amount of
    noise present.  It is argued that, if the data has power spectrum
    characterised by $1/f$ curve, we face a more severe problem, which
    cannot be solved inside the strict ICA model. This overlearning is
    better characterised by bumps instead of spikes. Both overlearning
    types are demonstrated in the case of artificial signals as well as
    magnetoencephalograms (MEG). Several methods are suggested to
    circumvent both types, either by making the estimation of the ICA
    model more robust or by including further modelling of the data.
altloc:
  - http://www.cis.hut.fi/jaakkos/papers/sarela03.pdf
  - http://www.jmlr.org/papers/volume4/sarela03a/sarela03a.pdf
chapter: ~
commentary: ~
commref: ~
confdates: ~
conference: ~
confloc: ~
contact_email: ~
creators_id:
  - 4715
creators_name:
  - family: Särelä
    given: Jaakko
    honourific: Mr
    lineage: ''
date: 2003-12
date_type: published
datestamp: 2004-05-24
department: ~
dir: disk0/00/00/36/38
edit_lock_since: ~
edit_lock_until: ~
edit_lock_user: ~
editors_id: []
editors_name:
  - family: Lee
    given: Te-Won
    honourific: prof.
    lineage: ''
  - family: Cardoso
    given: Jean-Francois
    honourific: prof.
    lineage: ''
  - family: Oja
    given: Erkki
    honourific: prof.
    lineage: ''
  - family: Amari
    given: Shun-Ichi
    honourific: prof.
    lineage: ''
eprint_status: archive
eprintid: 3638
fileinfo: /style/images/fileicons/application_pdf.png;/3638/1/sarela03.pdf
full_text_status: public
importid: ~
institution: ~
isbn: ~
ispublished: pub
issn: ~
item_issues_comment: []
item_issues_count: 0
item_issues_description: []
item_issues_id: []
item_issues_reported_by: []
item_issues_resolved_by: []
item_issues_status: []
item_issues_timestamp: []
item_issues_type: []
keywords: 'independent component analysis, blind source separation, overlearning, overfitting, spikes, bumps, high-order ICA'
lastmod: 2011-03-11 08:55:36
latitude: ~
longitude: ~
metadata_visibility: show
note: ~
number: ~
pagerange: 1447-1469
pubdom: FALSE
publication: Journal of machine learning research
publisher: MIT press
refereed: TRUE
referencetext: ~
relation_type: []
relation_uri: []
reportno: ~
rev_number: 12
series: ~
source: ~
status_changed: 2007-09-12 16:52:31
subjects:
  - comp-sci-stat-model
  - comp-sci-mach-learn
  - comp-sci-neural-nets
succeeds: ~
suggestions: ~
sword_depositor: ~
sword_slug: ~
thesistype: ~
title: ' Overlearning in marginal distribution-based ICA: analysis and solutions'
type: journalp
userid: 4715
volume: 4
<script src="//archive-bar.soton.ac.uk/archive-bar.js"></script>