[Eeglablist] ICA Misinformation

Arnaud Delorme arno at ucsd.edu
Sat Jun 17 10:42:05 PDT 2017


Dear Robert,

Thank you for your email and for your analysis. A few comments below:

1- I find it hard to believe all of the plots in your compressed document compared artifact free regions of data. For example, you observed 71% change over frontal channels in the delta band (see attached screen capture in the PDF document from PCT DIFF PRE VS POST-DELORME/PRE VS POST-DELORME_4.bmp). This is consistent with selecting portions of data which contains eye-related artifacts as I show in the rest of the attached PDF document.

2- The data you shared has a high density of blinks, there is only a handful segments of data where the subject stops blinking for more than 5 seconds. Even in these clean segments, in the frontal channels, we can see some slow some activity that likely reflect eye movement (or contamination from previous eye movements by the filter you might have used). ICA is able to remove these as well. I am attaching two examples in the PDF document. In between the two blinks below (see document), you can see that the “clean” segment is not that clean. It is still contaminated by eye artifacts at least on the first 3 channels (in order from top to bottom FP1,FP2, F7, F3, Fz, F4, F8). I do not think that anybody would argue that removing this activity amounts to removing brain activity. So it is important to be careful when selecting “clean” data segment.
3. I have removed all portions of data where eye activity was visible preserving only 40 seconds out of 410 in the original file and I am assessing below the data distortion. Spectrum was calculated on non-overlapping 1 minute data segments (I am attaching the script). I am attaching the scalp topography pre vs post in different frequency bands.

We can also see that the power at other channels is not affected. Beyond 5 Hz, the power spectrum does not seem to be affected even in frontal channels with very similar scalp topographies. In the attached document, I show the scalp topography pre vs post in different frequency bands. Note the difference in frontal channels at 1Hz and 5Hz. This is due to what is mentioned in 2, that even clean segments are not totally free of artifacts and I believe that these represent true eye artifacts removed by ICA. We can also see that the power at other channels is not affected. Beyond 5 Hz, the power spectrum does not seem to be affected even in frontal channels with very similar scalp topographies. There is no 71% difference in absolute power as in your plot (in my case, it seems to be about 25% at 1Hz over frontal channel and again, this correspond to removed residual eye blinks).

4. P-value need to be corrected for multiple comparisons. You have 19 channels and are looking at 20 frequency bands. That’s 380 t-test/p-value. By chance at the 5% threshold, 19 of these are going to be significant. This is an important limitation of the QEEG field by the way and the reason why QEEG papers rarely make it to reputable journals. And I have heard people say: well if I correct with classical Bonferoni, nothing shows up significant anymore, but there are less aggressive methods to correct for multiple comparison such as False Discovery Rate, the cluster and max method. The QEEG field should use these methods and they could easily be implemented in your software.

I am also attaching the script to reproduce my results from the raw EDF data file you shared (it in the PDF).
ICA is not a magical technique but it works pretty well for removing eye artifacts with the least of distortion of the signal. See for example this paper on real and simulated data (not from us http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0003004 <http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0003004>) concluding "ICA yielded almost perfect correction in all conditions.” There are many other such papers). And I would be the first to agree that there is a lot of subtleties of using ICA for artifact rejection and lot of potential limitations (instabilities in some conditions, issues with numerical precisions (double vs single), lack of exact reproducibility in some other cases because of the random initial conditions, multitude of algorithms). We have tried to address these issues over the years. Even in your dataset, it seems that Infomax ICA converge to at least 2 different solutions (I have run it 10 times on your data at least) yielding slightly different eye artifacts (both seem valid but differ at very low frequencies < 3Hz and very high frequencies above 60 Hz) and I will have to look at that in more details (I suspect this has to do with the stopping rule and the low number of channels in your dataset - I will run some tests and might have to change the default stop threshold for convergence). It remains that ICA is the best we have right now for artifact removal, and I am convinced that a poll of scientist in our field would show that more than 80% of EEG scientist agree with me.

I think it is better to rely on published data than argue about a particular dataset, although it is useful as well. At this point we may agree to disagree and EEG users can compare and contrast different approaches. They can download your data and use the script I provide to reproduce my results.

Best wishes,

Arno

PDF document
http://sccn.ucsd.edu/~arno/download/icaphaserebutal.pdf <http://sccn.ucsd.edu/~arno/download/icaphaserebutal.pdf>


> On Jun 15, 2017, at 12:32 PM, Robert Thatcher <rwthatcher2 at yahoo.com> wrote:
> 
> Dear Arnaud,
> 
>      I did statistical comparisons between 1 min & 40 seconds of artifact free EEG in the original EEG recording in with no eye movement artifact and the Win EEG ICA reconstruction and the ICA reconstruction that you did.  I compared two different artifact rejection methods used on the original EEG: 1- manual selections of artifact free data and, 2- the automatic template method of artifact rejection where I hand selected a 10 second sample of artifact free EEG and then used an algorithm that matched the peak-to-peak amplitudes of the 10 second template to the remainder of the record.   There were no statistically significant differences between these two artifact rejection methods.
> 
> Based on the time points of the artifact free data in the original EEG I selected the exact same time points in the Win EEG ICA reconstruction and in your ICA reconstruction.  Therefore all three data files contained 1 minute & 40 seconds of the same time points.   I then computed percent differences as well as paired t-tests between the original EEG and the two ICA reconstructions.   Here is a url to download the results:
> 
> http://www.appliedneuroscience.com/STATISTICS OF ARTIFACT FREE EEG VS POST ICA EEG.zip
> 
> As you can see there were very large statistically significant differences between the artifact free EEG in the original recording and the ICA reconstructions.   Your reconstruction was less distorted than the Win EEG reconstruction but both were significantly different than the original artifact free EEG.
> 
> I would be happy to send you the .edfs of the selected time points so that you can verify that the time points were identical and the original EEG did not have any eye movement artifact. 
> 
> These large magnitude of the differences between the original and unaltered data vs. the ICA altered data are similar to those that many WinEEG users find when they use the WinEEG ICA reconstruction method.   Therefore these large differences are not surprising and are commonly found especially when using the WinEEG ICA.  For example, Georges or Robert Lawson and others.
> 
> I also included screen captures of some of the waveforms showing visually detectable differences between the original and the ICA reconstruction using the WinEEG ICA.   The ICA that you used produced less visually obvious waveform changes but nonetheless there are some that are visually detectable.  However, the best way to understand the alterations of the artifact free sections is by JTFA and/or FFT and statistics.
> 
> Thank you for your patients in and dedication to exploring this important topic.  It is an important topic because of the obvious discrepancies that will exist in the scientific literature between simple deletion of artifact vs ICA reconstruction going forward.  Also because the entire EEG record is modified the ability to replicate findings is reduced when using ICA reconstruction.  Also, because there is some degree of decoupling between the underlying physiological origins of the EEG and a patient's brain then clinical correlation or effect size will be lower.
> 
> Best regards,
> 
> Robert
> 
> 
> 
> On Thursday, June 15, 2017, 10:11:45 AM EDT, Robert Thatcher <rwthatcher2 at yahoo.com> wrote:
> 
> 
> Hi Arno,
> 
> Thank you for your thoughtful post. 
> 
> “As far as phase distortion after removing ICA components (in my decomposition), I am not sure what you are referring to. Is it the minute shift when the red and black curve do not exactly superpose.”
> 
> I am referring to the differences in phase between pairs of EEG channels.   One can visually see differences in particular segments but it is best to use the Hilbert transform (cross-spectra) to compute instantaneous phase differences at any point in the record that one may want to average the absolute phase differences over some period of the record where there is no artifact and then conduct t-tests to evaluate the large effect sizes.   One can also compare the FFT spectra which is also an average, albeit more noisy.   The alteration of phase differences are present no matter what measure one uses.   The least reliable is a visual analysis although there are plenty of visual examples if one carefully reviews the traces.
> 
> "I would argue that the data after removing ICA artifacts reflect more brain activity than before, and that the minute shift is due to removal of small eye movement activity. I agree that this would have to be demonstrated, and that you cannot take my word for it.”
> Myself and many others do not disagree that elimination of artifact is important what we disagree with is the ICA reconstruction method that adulterates the artifact free segments of the record.  Why not simply delete the eye movement manually or like Neuroguide does with a signal detection algorithm that measures the voltage gradients produced by a blink or eye movement, etc?  In this way all of the original digital data samples are unaltered.
>  
> “The EEG signal is extremely noisy.”
> The vast number of EEG experts would disagree with you that the “EEG is extremely noisy”.   If this were true it would be obvious to every one with a total inability to replicate any EEG study and there would not be over 100,000 peer reviewed studies published in the National Library of Medicine.   Simply visually examine the EEG traces showing well behaved and well organized alpha rhythms or theta rhythms or beta rhythms which reflect large synchronous LFPs.
>  
> “The phase of the signal at one electrode site and one given time is not representative of the underlying brain signal.”
> This also cannot be true because the phase difference between electrodes and/or sources are produced by the physiological foundations of the brain and networks and are due to differences in synaptic rise times, synaptic integration times, differences in conduction velocity, etc.  This is the underlying brain signal and it is highly reproducible and clinically useful.  If your belief were valid then there would be no clinical correlations to the EEG such as schizophrenia or ADHD or depression or epilepsy or drug effects, etc.
>  
> “, if you have a picture of a star, would you rather remove a visual artifact that is 10-fold the size of your original signal or continue to look at your original signal (not being able to see much because of the large artifact masking most of it).”
> 
> This is an interesting take on my analogy and I agree that the 10-fold size artifact needs to be avoided or eliminated but not by using ICA reconstruction that effects the artifact free parts of the spectrum and thereby distorts the measurement not only of the one star that you are looking at but also all other stars and planets in the universe.
> 
>  
> “Even if ICA was introducing minute distortion in phase”
> 
> I wish that the distortion in phase difference was “minute” but the fact is that it is large and easily demonstrated as it has been by numerous scientists/clinicians over the last few years.  For example, t-tests between the artifact free segments in the original EEG vs. the ICA reconstructed new time series are mostly significant at P < 0.00001.   I will do some additional statistical comparisons so that you can better understand the large effect sizes of ICA phase difference distortion.  
> 
> This is an important dialog and I appreciate your dedication and willingness to consider these issues.  
> 
>  
> Best wishes,
> 
>  
> Robert
> 
> 
> 
> On Wednesday, June 14, 2017, 11:49:54 PM EDT, Arnaud Delorme <arno at ucsd.edu> wrote:
> 
> 
> Dear Robert,
> 
> There is no need to remove more components - except maybe for temporal muscles components (I would have to look again at your data to see if I can identify any). The procedure is to identify a handful of artifact components, remove them and then your data is cleaned of these artifacts. I personally rarely identify more than 4 artifact component in a given subject (some other researchers have a more aggressive approach and remove more). I like to remove components I am sure of.
> 
> ICA is a linear decomposition that isolate sources which are maximally independent. Blinks are mostly independent of brain activity (on first approximation) so ICA is able to isolate them.
> 
> "You already showed that two ICA component removals results in more phase distortion than the removal of one ICA component.” In all of our exchanges I have always removed 2 components. I have never removed one ICA component.
> 
> As far as phase distortion after removing ICA components (in my decomposition), I am not sure what you are referring to. Is it the minute shift when the red and black curve do not exactly superpose. I have two comments on that.
> 
> I would argue that the data after removing ICA artifacts reflect more brain activity than before, and that the minute shift is due to removal of small eye movement activity. I agree that this would have to be demonstrated, and that you cannot take my word for it.
> 
> "Even a small amount of adulteration or distortion of EEG phase differences is not good and must be avoided at all costs. This is analogous to the use of telescopes that measure phase differences in the spectrum from stars moving in the universe."
> 
> The data we are looking at on the scalp is a summation of millions of neuron activity and the phase we are observing a cumulative average of this signal (pondered by the geometry of the brain, difference in conductivity of different tissues etc...). The EEG signal is extremely noisy. The phase of the signal at one electrode site and one given time is not representative of the underlying brain signal. Even if ICA was introducing minute distortion of the "true phase" at given channels, properly removing artifacts (which are 10-fold the amplitude of brain EEG signal) like ICA does is more important than preserving the exact phase at a given time. In your analogy of looking at stars, if you have a picture of a star, would you rather remove a visual artifact that is 10-fold the size of your original signal or continue to look at your original signal (not being able to see much because of the large artifact masking most of it). Even if ICA was introducing minute distortion in phase (which I do not believe it does because it deals with instantaneous mixtures) , it is worth it given the advantage it provides.
> 
> The exact phase at one electrode site is not informative in itself. Differences in phase between 2 electrode sites is not informative either because there may be dozens of possibility for activity within the brain to generate such phase difference. One must move to the source level, and this is what ICA is doing (although see also my previous message).
> 
> Best wishes,
> 
> Arno
> 
> 
>> On Jun 14, 2017, at 6:50 PM, Robert Thatcher <rwthatcher2 at yahoo.com <mailto:rwthatcher2 at yahoo.com>> wrote:
>> 
> 
> Arnaud,
>    It is interesting to see of the amount of distortion of phase differences of the original artifact free segments of the EEG record by ICA is a linear or nonlinear function of the number of ICA components that are removed to do the reconstruction of a different time series.  You already showed that two ICA component removals results in more phase distortion than the removal of one ICA component.  If you were to remove three and then reconstruct and then four and then five, etc and reconstruct and then attach the  .edf files and share them with the forum then we can plot the magnitude of phase distortion of the artifact free sections of the original record due to the ICA reconstructions.   Based on embedding theory one would expect a linear relationship but there may be a nonlinear relationship with an asymptote at about two removals given there are only 19 channels.
> 
> Please try this experiement with one or more EEG dataset, the one that was produced by ICA reconstruction in Australial is a good starting point but it will be good to do this experiment with two or three other EEG recordings.
> 
> Thank you for honest interest in exploring the extent of phase difference distortion by ICA so that we can better understand it.
> 
> Best regards,
> 
> Robert 
> 
> On Wednesday, June 14, 2017, 9:00:58 PM EDT, Robert Thatcher <rwthatcher2 at yahoo.com <mailto:rwthatcher2 at yahoo.com>> wrote:
> 
> 
> Arnaud,
>     It does not make any difference which components that the scientist/clinicians removed because your own analyses confirmed phase difference distortion by ICA when your removed your own components.  Please try different ICA component removal and attach the edf files to see if you can create a reconstruction of the time series that DOES NOT distort or corrupt the phase differences between channels in the original EEG recording.  Up to this point in time you have resoundeding proven that ICA reconstruction oes distort phase differences no matter what reconstruction is used.
> 
> It is important to recognize and to pubically accept that phase or time differences between channels in the EEG is due to physiological processes like differences in synaptic rise times, differences in synaptic summation times and differences in conduction velocities, etc.   Even a small amount of adulteration or distortion of EEG phase differences is not good and must be avoided at all costs.
> 
> This is analogous to the use of telescopes that measure phase differences in the spectrum from stars moving in the universe.  If ICA were used to distort the phase differences in the spectrum measured by telescopes because one believes that all telescopes have artifact then we would not know huge amounts about the nature and future of the universe.  The same is true for the human EEG.
> 
> Bob
> 
> 
> On Wednesday, June 14, 2017, 7:23:43 PM EDT, Arnaud Delorme <arno at ucsd.edu <mailto:arno at ucsd.edu>> wrote:
> 
> 
>>    Thank you for attaching your ICA reconstructed edf file.  It involved removal of two ICA components and the magnitude of changes in phase differences between channels is greater than the one provided by the scientists/clinicians in Australia that deleted only one ICA component.  This is consistent with Taken's theorum and also differential geometry theorums dealing with manifold mapings and Lie groups etc.  I know for certain that they used ICA and not PCA.
> 
> Yes, I meant that the data is usually preprocessed by PCA before doing ICA in commercial softwares, which could be the problem (although I do not think it was in that case). We would need to see which components were removed.
> Best wishes,
> 
> Arno
> 
> 
>> ICA is excellent in feature detection and the brain operates by highly efficient sub-clusters of neurons extracting features, e.g., face recognition by combining features like eye brows, head shape, ears, chin, etc 
>> 
>> Here is a url to a recent study showing that only 206 neurons are necessary to encode face recognition in monkeys:
>> 
>> http://dx.doi.org/10.1016/j.cell.2017.05.011 <http://dx.doi.org/10.1016/j.cell.2017.05.011>
>> 
>> The individual face components are like ICA face components for face recognition.  However, the anterior temporal lobes are just one node among many nodes in a network so that the monkey can make the correct adaptive decisions in very short periods of time by network coherence and phase locking and phase shifting with other nodes in networks.
>> 
>> The problem with ICA is in its use in artifact rejection and then reconstruction of a new time series that results in a new time series that is disconnected from brain network connectivity dynamics of phase shift and phase lock and coherence and cross-frequency coupling and phase amplitude coupling, etc.
>> 
>> Thank you again and lets continue to seek answers to how best to use ICA for network dynamics without adulterating the original time and phase relations between parts of the brain.
>> 
>> Robert
>> 
>> 
>> On Wednesday, June 14, 2017, 6:21:32 PM EDT, Arnaud Delorme <arno at ucsd.edu <mailto:arno at ucsd.edu>> wrote:
>> 
>> 
>> Hi Robert,
>> 
>>> The Australian data was analyzed by two scientists/clinicians in the audience of a workshop that I was doing in 2014 and they are the ones that did the ICA component selection using commercial WinEEG software and not me.
>> 
>> Most commercial EEG software preprocess the data using PCA to reduce the dimensionality of the data. The idea behind this is that users should not have to go through as many components as channels. It is easier to have them select components within 5 or 10 exemplars. However this PCA dimension reduction can bias the reconstruction (we have data to back this up but it is not published yet).
>> 
>> However, I do not think PCA dimension reduction before running ICA was responsible for what you observed (because your data is very clean and even after PCA and the 2 artifact components have huge contribution to the data variance, you would get very similar components). I think the WinEEG users simply did not select the correct artifact components, or maybe WinEEG failed to implement ICA correctly.
>> 
>>> You are welcome to download NeuroGuide and install and launch and then paste the key A into an email to me.  I have posted a tutorial on our webpage but I can create a better tutorial to reduce the learning curve.  Similarly when I am able to concentrate on EEGlab then you can tutor me to reduce my learning curve.  Here is a url to the download webpage:
>>> http://www.appliedneuroscience.com/Download_NeuroGuide.htm <http://www.appliedneuroscience.com/Download_NeuroGuide.htm>
>>> 
>>> At the end of the day together lets find ways to use the full power of ICA to explore network dynamics which is my favorite topic and also one that future science depends on.
>> 
>> Yes, I agree on that view. Exploring network dynamics with ICA is not an easy topic. The trend these days is not to use ICA for connectivity analysis but instead define regions of interest and compute pairwise connectivity between all brain regions as in this recent paper https://www.ncbi.nlm.nih.gov/pubmed/28300640 <https://www.ncbi.nlm.nih.gov/pubmed/28300640>. What can be done is to use ICA components to define these regions and compute activity in these regions. It is an open area of research.
>> 
>> Best wishes,
>> 
>> Arno
>> 
>>> 
>>> Best regards,
>>> 
>>> Robert
>>> 
>>> 
>>> On Wednesday, June 14, 2017, 4:40:43 PM EDT, Arnaud Delorme <arno at ucsd.edu <mailto:arno at ucsd.edu>> wrote:
>>> 
>>> 
>>> Dear Robert,
>>> 
>>> There does seem to be a phase difference in your powerpoint. However, it is important to know which ICA component you removed to understand why this is the case. Are you sure these were artifactual components? Removing brain components may alter the phase of the signal recorded on the scalp (it would be as if you were removing from the scalp signal the contribution of a brain area). Without that information, it is not possible to figure out the origin of the phase difference. 
>>> 
>>> This seems to be the same data you shared yesterday. I have looked at it. Black is before ICA and red after removing the 2 eye components. You can see that there is no phase shift at 102.43 second after I remove the two artifactual ICA components. I have provided the code in my email yesterday if you want to reproduce this result in EEGLAB.
>>> 
>>> Best wishes,
>>> 
>>> Arno
>>> 
>>> <ICA_phase_example.png>
>>> 
>>>> On Jun 14, 2017, at 11:14 AM, Robert Thatcher <rwthatcher2 at yahoo.com <mailto:rwthatcher2 at yahoo.com>> wrote:
>>>> 
>>>> <Example of Phase Differences at   1min & 46 seconds.pptx>
>>> 
>>> 
>>> <ICA_phase_example.png>
>> 
>> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20170617/384f672e/attachment-0001.html>


More information about the eeglablist mailing list