[Eeglablist] Inconsistent results using clean_artifacts

[Gil Avila, Cristina]

Dear Prabhakar and Makoto,

Thank you for your responses. I have tried both
1) Uncomment the lines in clean_channels() that fix the seed for random stream and
2) Keep the clean_channels() code as it as and fix in my code the seed of the random number generator: rng('default');

In both cases I get a stable set of bad channels although the set of bad channels in 1) is different than the set of bad channels in 2). These 'tricks' make stable the bad channel detection. Nevertheless, I am concerned about the RANSAC algorithm itself. I believe it is suboptimal that, depending on how the channels were initially chosen,  different sets of bad channels are given as an output. I don't know if this will have an impact on later results such as power effects between groups. Do you think that this channel discrepancy will impact later results or do you think that it will be diluted when looking into large datasets?

Best,
Cristina

-----Original Message-----
From: Makoto Miyakoshi <mmiyakoshi at ucsd.edu> 
Sent: Thursday, 3 March 2022 18:40
To: eeglablist at sccn.ucsd.edu
Subject: Re: [Eeglablist] Inconsistent results using clean_artifacts

Dear Cristina,

Sorry for the inconvenience.

> I am using the last version of clean_rawdata plugin and every time 
> that I
run clean_rawdata on the same dataset I get a different set of 'bad channels'. Is this problem fixed?

According to this page, clean_channels() was updated 2 years ago. Without testing, I guess it is probably not fixed.
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_sccn_clean-5Frawdata&d=DwIFaQ&c=-35OiAkTchMrZOngvJPOeA&r=kB5f6DjXkuOQpM1bq5OFA9kKiQyNm1p6x6e36h3EglE&m=8vkQdVejMdf_IHSTIf0e5McIIPD3TbqIxIXVpdRjCZxECTENaMdkB4EoFHSDinQX&s=yz8W2rwlcw_xyQhxNND6QqqHF0ahjP6l7vk_BsP5sgw&e= 

If I remember correctly, it uses RANSAC which uses 'repeated random subsampling'. The unstable output is a nature of the algorithm and it cannot be 'fixed'.
I know this is an annoying issue. We always prefer to use deterministic approaches for the purpose of reproducibility.

Here are my suggested work arounds.
1. This is a little bit technical, but you can fix the seed of a random number generator by using rng(). Theoretically, this guarantees the complete reproducibility from the algorithm. In clean_channels() line 171, you can see the following line commented out. I don't know why it is commented out, but if you can fix the seed for the random number generator there, it may work out. I would try rng() instead of what is there now.
%stream = RandStream('mt19937ar','Seed',435656);

2. Alternatively, you disable the electrode removal function in
clean_rawdata() and perform your own electrode rejection process. Sometimes even a simple max value approach like the one used in trimOutlier() is sufficient, so you may try it out.

Makoto


On Wed, Mar 2, 2022 at 9:01 AM Gil Avila, Cristina via eeglablist < eeglablist at sccn.ucsd.edu> wrote:

> I would like to reopen a topic from 2020 regarding the consistency of 
> clean_artifacts. I am using the last version of clean_rawdata plugin 
> and every time that I run clean_rawdata on the same dataset I get a 
> different set of 'bad channels'. Is this problem fixed? Specifically I am running:
>
> EEG =
> pop_clean_rawdata(EEG,'FlatlineCriterion',5,'ChannelCriterion',0.8,'Li
> neNoiseCriterion',4,'Highpass',[0.25
> 0.75],'BurstCriterion','off','WindowCriterion','off','BurstRejection',
> 'off','Distance','Euclidian','WindowCriterionTolerances','off');
>
> Best,
> Cristina Gil
> --
> Cristina Gil Ávila - PhD candidate
> Department of Neurology
> Technische Universität München
> Munich, Germany
> cristina.gil at tum.de<mailto:cristina.gil at tum.de>
> painlabmunich.de<
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.painlabmunich
> .de_&d=DwIFAw&c=-35OiAkTchMrZOngvJPOeA&r=kB5f6DjXkuOQpM1bq5OFA9kKiQyNm
> 1p6x6e36h3EglE&m=c_kRevMMCP43N5DUC96kTPQlI81ezyGf6D0jCBCbtNsjpI0SNRrFx
> tNyOi0J1P05&s=nZ9fhNHv4Zt2tPe1W18k_LDsgbW7YJcgl9PBRpNInAk&e=
> >
>
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to 
> eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to 
> eeglablist-request at sccn.ucsd.edu
>