[Eeglablist] Double Dipping with ICA

[Scott Makeig]

Re the double-dipping example Makoto provides: Don't neglect to check,
however, whether and to what extent IC ERPs do contribute to the ICs signal
variance - this may be less than 1% (i.e., removing the IC ERP from every
trial reduces the overall IC signal variance by ~1%). If you wish, you can
remove the ERPs from every trial after clustering, and then test
differences in remaining cluster variance-not-explained-by-the-ERPs. Or,
you can normalize the ERPs *before* clustering based on equivalent dipole
location and normed ERPs, then test differences in cluster
variance/power/etc.

Scott

On Sun, Nov 8, 2020 at 1:39 PM Makoto Miyakoshi <mmiyakoshi at ucsd.edu> wrote:

> Dear Nick,
>
> Two things.
>
> 1. The second ICA you mentioned does not change the result. Just try it and
> you see what happens.
>
> 2. Double dipping refers to a procedure that is known to inflate Type I
> error. ICA uses mutual information, and you test amplitude/phase
> metric--these two do not have apparent correlation. If you say, cluster ICs
> by ERP waveforms, then perform ERP amplitude difference, this is double
> dipping (why? because IC cluster's ERP variance is minimized by clustering,
> but this variance information is being used in the subsequent statistical
> test)
>
> Given that,
>
> > Are there reasons not to do this or to do it?
>
> You may do so but it is meaningless because the second ICA in that case
> does nothing. And this has nothing to do with the double dipping problem.
>
> Makoto
>
> On Sun, Nov 8, 2020 at 8:22 AM Nicholas Dogris <
> drdogris at neurofieldneurotherapy.com> wrote:
>
> > Will my data be distorted if I run ICA, remove IC's such as eye blinks,
> and
> > then run ICA again on the same dataset?  Are there reasons not to do this
> > or to do it?
> >
> > Cheers,
> >
> > Nick
> > _______________________________________________
> > Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> > To unsubscribe, send an empty email to
> > eeglablist-unsubscribe at sccn.ucsd.edu
> > For digest mode, send an email with the subject "set digest mime" to
> > eeglablist-request at sccn.ucsd.edu
> >
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to
> eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to
> eeglablist-request at sccn.ucsd.edu
>


-- 
Scott Makeig, Research Scientist and Director, Swartz Center for
Computational Neuroscience, Institute for Neural Computation, University of
California San Diego, La Jolla CA 92093-0559, http://sccn.ucsd.edu/~scott