<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
Dear Andreas, dear Makoto,
<p><br>
</p>
<p>I have re-run the pre-processing routine and ICA for one of the affected subjects with consistent Matlab version (R2015b) and still get the same results in terms of runtime (>40 h / 14.5 h / < 1h depending on the previously used high-pass filter). Thus,
the problems don’t seem to have been caused by inconsistent versions.</p>
<p><br>
</p>
<p>Thank you Makoto for your suggestion, I haven’t used the trimOutlier() plugin so far - I will try to check for outliers that way. If bad data quality was the reason, shouldn’t the long runtimes (in a specific subject) occur irrespective of the used high-pass
filter?</p>
<p><br>
</p>
<p>One aspect I noticed is that the data format (EEG.data) is single precision – could this be the problem? I’ve just read in the PREP pipeline paper that double precision computation is essential for filtering; it is mentioned that the
<em>eeg_checkset</em> function converts EEG data to single precision per default and one should override this default by changing the eeglab settings (<em>pop_editoptions,
</em>set<em> option_single </em>to false). I have changed the eeglab options following the instructions on the eeglab wiki page ('File' -> 'Memory and other options' -> 'If set, use single precision under...' uncheck it). In my understanding this should be
the same, right?</p>
<p><br>
</p>
<p>I appreciate your offer to try replicate the problem. I am not sure though what I would have to make available to you - would the pre-processed dataset(s) of one affected subject be sufficient? Also, do you need to be able to actually run my scripts or would
it be enough to see the relevant parts of the code? (because in the main scripts I also retrieve information from additional files and call custom-made functions to process co-registered eye-tracking data…).</p>
<p><br>
</p>
<p>Thanks a lot for your effort,<br>
</p>
Hannah
<p><br>
</p>
<p><br>
</p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="margin:0px; font-family:Calibri,Arial,Helvetica,sans-serif">
<div class="BodyFragment"><font size="2">
<div class="PlainText">Hannah Hiebel, Mag.rer.nat.<br>
Cognitive Psychology & Neuroscience</div>
<div class="PlainText">Department of Psychology, University of Graz</div>
<div class="PlainText">Universitätsplatz 2, 8010 Graz, Austria<br>
</div>
</font></div>
</div>
</div>
<div style="word-wrap:break-word">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>Von:</b> Andreas Widmann <widmann@uni-leipzig.de><br>
<b>Gesendet:</b> Donnerstag, 19. Jänner 2017 21:53<br>
<b>An:</b> Hiebel, Hannah (hannah.hiebel@uni-graz.at)<br>
<b>Cc:</b> eeglablist@sccn.ucsd.edu<br>
<b>Betreff:</b> Re: [Eeglablist] ICA running very slowly</font>
<div> </div>
</div>
<div>Hi Hannah,
<div class=""><br class="">
</div>
<div class="">I would like to try to replicate this behavior. Could you please make available one of the affected datasets and the relevant parts of the code used for pre-processing and ICA, e.g. via the bugtracker or Dropbox? Are there possibly data discontinuities
without boundary markers? Did you keep MATLAB version constant?</div>
<div class=""><br class="">
</div>
<div class="">Best,</div>
<div class="">Andreas</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">Am 19.01.2017 um 09:41 schrieb Hiebel, Hannah (<a href="mailto:hannah.hiebel@uni-graz.at" class="">hannah.hiebel@uni-graz.at</a>) <<a href="mailto:hannah.hiebel@uni-graz.at" class="">hannah.hiebel@uni-graz.at</a>>:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">Dear Alberto and Tarik,</span></div>
<p class="MsoNormal" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class=""> </span></p>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">thank you very much for your suggestions. I work on a computer with i7 3.60 GHz processor, 8 GB RAM or notebook with i7 2.5 GHz and 8GB Ram – this should be okay.</span></div>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">Gladly, the ICA eventually finds a solution and the IC maps look good. However, the question for me is still why does the ICA become >10 times slower after changing the pre-processing routine. I’ve continued testing and indeed the high-pass filter
seems to be responsible for the differences.</span></div>
<p class="MsoNormal" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class=""> </span></p>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">In my recent routine I used the eeglab windowed sinc FIR filter with 1 Hz cut-off frequency, 1 Hz transition bandwidth, 0.001 passband ripple, Kaiser window. When I change the filter (settings) while keeping all other steps the same, I see huge
differences in ICA runtime in some subjects. That is, when using a 0.1 Hz Butterworth filter instead, ICA is running fast again (< 1h for the subjects where it took > 30h before). With the eeglab basic FIR filter with 1 Hz passband edge and default settings
defined by the internal heuristic (resulting in 0.5 Hz cut-off, 1 Hz trans. bandwidth) it’s also running much faster in most subjects but already takes >20h in the “problematic” cases.<span class="Apple-converted-space"> </span></span></div>
<p class="MsoNormal" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class=""> </span></p>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">This gives me the impression that the higher cut-off frequency causes the problems (or maybe stopband edge and attenuation are more decisive?).</span></div>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">That's very surprising as I would not have expected the filter to have such an impact and a higher cut-off is normally recommended.</span></div>
<p class="MsoNormal" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class=""> </span></p>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">I’d be very grateful if anyone could provide more insight!</span></div>
<p class="MsoNormal" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class=""> </span></p>
<div class="" style="margin-top:0px; margin-bottom:0.0001pt; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); line-height:normal">
<span class="">Best,<br class="">
Hannah</span><span class=""></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<br class="">
</div>
<div class="" style="margin-top:0px; margin-bottom:0px; font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<br class="">
</div>
<div id="Signature" class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<div name="divtagdefaultwrapper" class="" style="margin:0px; font-family:Calibri,Arial,Helvetica,sans-serif">
<div class="BodyFragment"><font class="" size="2">
<div class="PlainText">Hannah Hiebel, Mag.rer.nat.<br class="">
Cognitive Psychology & Neuroscience</div>
<div class="PlainText">Department of Psychology, University of Graz</div>
<div class="PlainText">Universitätsplatz 2, 8010 Graz, Austria<br class="">
<br class="">
</div>
</font></div>
</div>
</div>
<div class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); color:rgb(33,33,33)">
<hr tabindex="-1" class="" style="display:inline-block; width:565.453125px">
<div id="divRplyFwdMsg" dir="ltr" class=""><font class="" style="font-size:11pt" face="Calibri, sans-serif"><b class="">Von:</b><span class="Apple-converted-space"> </span>Alberto Sainz <<a href="mailto:albertosainzc@gmail.com" class="">albertosainzc@gmail.com</a>><br class="">
<b class="">Gesendet:</b><span class="Apple-converted-space"> </span>Mittwoch, 18. Jänner 2017 04:29<br class="">
<b class="">An:</b><span class="Apple-converted-space"> </span>Hiebel, Hannah (<a href="mailto:hannah.hiebel@uni-graz.at" class="">hannah.hiebel@uni-graz.at</a>)<br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:eeglablist@sccn.ucsd.edu" class="">eeglablist@sccn.ucsd.edu</a><br class="">
<b class="">Betreff:</b><span class="Apple-converted-space"> </span>Re: [Eeglablist] ICA running very slowly</font>
<div class=""> </div>
</div>
<div class="">
<div dir="ltr" class="">I would suggest to try in a different computer. I have been applying ICA in a 14 electrode 30min continuous EEG recording (around 40mb) in two different computers. 2Ghz dual core computer took 1h. 2.2Ghz i7 takes around 5 minutes.<br class="">
<br class="">
I know your data is larger but just to say that the processor (and probably the RAM if is too small) matters a lot.<br class="">
<br class="">
Good luck</div>
<div class="gmail_extra"><br class="">
<div class="gmail_quote">2017-01-16 20:26 GMT+01:00 Hiebel, Hannah (<a href="mailto:hannah.hiebel@uni-graz.at" class="">hannah.hiebel@uni-graz.at</a>)<span dir="ltr" class=""><<a href="mailto:hannah.hiebel@uni-graz.at" target="_blank" class="">hannah.hiebel@uni-graz.at</a>></span>:<br class="">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left-width:1px; border-left-color:rgb(204,204,204); border-left-style:solid; padding-left:1ex">
<div dir="ltr" class="" style="font-size:12pt; background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif">
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">Dear all,<br class="">
</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
<span class="" style="font-size:9pt; font-family:Verdana,sans-serif"></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">I am using ICA to clean my EEG data for eye-movement related artifacts. I’ve already done some testing in the past to see how certain
pre-processing steps affect the quality of my decomposition (e.g. filter settings). In most cases, it took approximately 1-2 hours to run ICA for single subjects (62 channels: 59 EEG, 3 EOG channels).<br class="">
</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
<span class="" style="font-size:9pt; font-family:Verdana,sans-serif"></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">Now that I run ICA on my final datasets it suddenly takes hours over hours to do only a few steps. It still works fine in some subjects
but in others runica takes up to 50 hours. I observed that in some cases the weights blow up (learning rate is lowered many times); in others it starts right away without lowering the learning rate but every step takes ages.</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">I’ve done some troubleshooting to see if a specific pre-processing step causes this behavior but I cannot find a consistent pattern.
It seems to me though that (at least in some cases) the high-pass filter played a role – can anyone explain how this is related? Could a high-pass filter potentially be too strict?<br class="">
</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
<span class="" style="font-size:9pt; font-family:Verdana,sans-serif"></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">On the eeglablist I could only find discussions about rank deficiency (mostly due to using average reference) as a potential reason.
I re-referenced to linked mastoids – does this also affect the rank? When I check with<span class="Apple-converted-space"> </span><em class=""><span class="" style="font-family:Verdana,sans-serif">rank(EEG.data(:, :))<span class="Apple-converted-space"> </span></span></em>it
returns 62 though, which is equal to the number of channels. For some of the “bad” subjects I nonehteless tried without re-referencing – no improvement. Also, reducing dimensionality with pca ("pca, 61") didn’t help.<br class="">
</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
<span class="" style="font-size:9pt; font-family:Verdana,sans-serif"></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">Any advice would be very much appreciated!<br class="">
</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
<span class="" style="font-size:9pt; font-family:Verdana,sans-serif"></span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">Many thanks in advance,</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><span class="" style="font-size:9pt; font-family:Verdana,sans-serif">Hannah</span></div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
</div>
<div class="" style="margin-top:0px; margin-bottom:0px"><br class="">
</div>
<div id="m_-9147620952188609437Signature" class="">
<div name="divtagdefaultwrapper" class="" style="margin:0px; font-family:Calibri,Arial,Helvetica,sans-serif">
<div class="m_-9147620952188609437BodyFragment"><font class="" size="2">
<div class="m_-9147620952188609437PlainText">Hannah Hiebel, Mag.rer.nat.<br class="">
Cognitive Psychology & Neuroscience</div>
<div class="m_-9147620952188609437PlainText">Department of Psychology, University of Graz</div>
<div class="m_-9147620952188609437PlainText">Universitätsplatz 2, 8010 Graz, Austria</div>
</font></div>
</div>
</div>
</div>
<br class="">
______________________________<wbr class="">_________________<br class="">
Eeglablist page:<span class="Apple-converted-space"> </span><a href="http://sccn.ucsd.edu/eeglab/eeglabmail.html" rel="noreferrer" target="_blank" class="">http://sccn.ucsd.edu/eeglab/<wbr class="">eeglabmail.html</a><br class="">
To unsubscribe, send an empty email to<span class="Apple-converted-space"> </span><a href="mailto:eeglablist-unsubscribe@sccn.ucsd.edu" class="">eeglablist-unsubscribe@sccn.<wbr class="">ucsd.edu</a><br class="">
For digest mode, send an email with the subject "set digest mime" to<span class="Apple-converted-space"> </span><a href="mailto:eeglablist-request@sccn.ucsd.edu" class="">eeglablist-request@sccn.ucsd.<wbr class="">edu</a><br class="">
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
<span class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); float:none; display:inline!important">_______________________________________________</span><br class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<span class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); float:none; display:inline!important">Eeglablist
page: <a href="http://sccn.ucsd.edu/eeglab/eeglabmail.html" class="">http://sccn.ucsd.edu/eeglab/eeglabmail.html</a></span><br class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<span class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); float:none; display:inline!important">To
unsubscribe, send an empty email to <a href="mailto:eeglablist-unsubscribe@sccn.ucsd.edu" class="">
eeglablist-unsubscribe@sccn.ucsd.edu</a></span><br class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255)">
<span class="" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:16px; font-style:normal; font-weight:normal; letter-spacing:normal; orphans:auto; text-align:start; text-indent:0px; text-transform:none; white-space:normal; widows:auto; word-spacing:0px; background-color:rgb(255,255,255); float:none; display:inline!important">For
digest mode, send an email with the subject "set digest mime" to <a href="mailto:eeglablist-request@sccn.ucsd.edu" class="">
eeglablist-request@sccn.ucsd.edu</a></span></div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</body>
</html>