[Eeglablist] Is CleanLine working?

Tue May 5 17:28:20 PDT 2020

As an example of CleanLine 1.04 bugs, I found one when running raw EEG through it.  Basically rmlinesmovingwinc expects data matrices to be time x chans whereas EEGlab standard is chans x time.  The rmlinesmovingwinc function runs the data through a change_row_to_column function but it only operates if the data is a vector or if it is a structured variable.  This data mismatch results in a crash.

Conversely, when I tried the Prep version, the bug had been fixed.  With some real data I got substantially better results as well (once I fixed the aforementioned bug).  It is also parallelized for those with the Parallel Computing Toolbox.  It seems to be pretty much CleanLine 2.0; that is, the same core code and approach but with substantial bugfixes and optimizations.  Arno, perhaps you might consider just taking CleanLine offline in favor of the Prep version?

Cheers!

Joe

> On Apr 25, 2020, at 22:53, Delorme, Arnaud <adelorme at ucsd.edu> wrote:
> 
> Hi Makoto,
> 
> These algorithms should be tested on real data with real line noise, not perfect sinusoidal signal.
> The power of cleanline is to allow removing line noise that will shift in phase, which was not the case in your simulation.
> 
> Take 20 EEG datasets and see how these algorithms perform. Also I would not switch to one or the other without understanding the difference between them.
> 
> Arno
> 
>> On Apr 25, 2020, at 5:44 PM, Makoto Miyakoshi <mmiyakoshi at ucsd.edu> wrote:
>> 
>> Dear Robert, Saraswati, Malte, Ekansh, and all the curious CleanLine users,
>> 
>> I ran the test. The results are uploaded in this page.
>> https://sccn.ucsd.edu/wiki/Makoto's_preprocessing_pipeline#Validation_.2804.2F25.2F2020_added.29
>> 
>> 
>> Conclusion is that I should switch to cleanLineNoise that is a part of PREP
>> Pipeline. I will update my preprocessing pipeline soon.
>> If you need to do this process asap, you can download my test code uploaded
>> there and copy and paste the relevant part into your batch code. Of course,
>> you have to install PREP Pipeline first!
>> 
>> Ekansh, I could not replicate your result. But I still think it was good
>> that I tested it with pink noise just to make sure that the denoising
>> behavior is not affected by the background noise. Thank you for testing it
>> together. Your report motivated me to do it.
>> 
>> Makoto
>> 
>> On Sat, Apr 25, 2020 at 3:41 AM Ekansh Sareen <ekansh15139 at iiitd.ac.in>
>> wrote:
>> 
>>> Dear Makoto,
>>> 
>>> As per your earlier mail, I created a 1-minute single-channel signal
>>> sampled at 128Hz consisting of only 60Hz sine wave. I ran Cleanline over
>>> this signal to remove the 60Hz sine wave and your concern was right. It did
>>> not remove the 60Hz sine wave, in fact, it was left unaffected. However, I
>>> tried doing this again by adding a 10Hz sine wave to the 60Hz pure sine
>>> wave and again ran the Cleanline to remove just the 60Hz pulse. This time
>>> Cleanline successfully removed the 60Hz sine wave and preserved the 10Hz
>>> component.
>>> Note- I have used the latest available version of the Cleanline for this.
>>> Further, To evaluate the performance of Cleanline, I compared the FFT
>>> response of the signal before and after Cleanline.
>>> 
>>> In my opinion, as EEG data is always mixed with multiple frequency
>>> components, removing a 60/50Hz line noise using Cleanline is adequate and
>>> should provide reliable results (In fact it does!). Irrespective of this,
>>> Cleanline should have been able to remove the pure 60Hz sine wave and
>>> return a flat line which it doesn't seem to achieve. Did you receive any
>>> information specific to this concern from the developers?
>>> 
>>> Regards,
>>> Ekansh Sareen
>>> 
>>> On Sat, Apr 25, 2020 at 3:50 AM Makoto Miyakoshi <mmiyakoshi at ucsd.edu>
>>> wrote:
>>> 
>>>> Dear list members,
>>>> 
>>>> I got information from developers that CleanLine implemented in PREP
>>>> Pipeline has fixed bugs from the original CleanLine.
>>>> Actually I was surprised to find out that PREP does not use the original
>>>> CleanLine files but the implemented functions seemed heavily re-organized
>>>> even if using a large part of the original CleanLine code.
>>>> 
>>>> Is anyone interested in testing the performance difference?
>>>> 
>>>> Makoto
>>>> 
>>>> 
>>>> 
>>>> On Tue, Jan 14, 2020 at 2:36 PM Makoto Miyakoshi <mmiyakoshi at ucsd.edu>
>>>> wrote:
>>>> 
>>>>> Dear list members,
>>>>> 
>>>>> I have never been satisfied with the result with CleanLine. In fact, I
>>>>> think it is not working as it should be.
>>>>> 
>>>>> Can anyone run the following test and share the results? I would greatly
>>>>> appreciate it.
>>>>> 
>>>>> 1) Prepare 1-min long 1-ch data with 60Hz pure sine wave.
>>>>> 2) Find CleanLine parameters that can completely remove this 60Hz sine
>>>>> wave so that it returns a flat line as a result.
>>>>> 
>>>>> I doubt if we can achieve this. Can we? But if it cannot pass this
>>>> simple
>>>>> test, what's the point...  and what's the best result is like as of now?
>>>>> 
>>>>> Makoto
>>>>> 
>>>> _______________________________________________
>>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>>>> To unsubscribe, send an empty email to
>>>> eeglablist-unsubscribe at sccn.ucsd.edu
>>>> For digest mode, send an email with the subject "set digest mime" to
>>>> eeglablist-request at sccn.ucsd.edu
>>>> 
>>> 
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
> 
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu

--------------------------------------------------------------------------------

Joseph Dien, PhD
Senior Research Scientist
Department of Human Development and Quantitative Methodology
University of Maryland, College Park
https://urldefense.com/v3/__http://joedien.com__;!!Mih3wA!WMj1Avz_s6BZ6-5MecyoXFus57YK2HZvAoNGQ0xOPI2IPduYl0qhAkXv_MUZgQSZXmK7KQ$ 