<div dir="ltr"><div><div><div>Re: About running ica on several machines (Tarik S Bel-Bahar)<br><br></div>Dear Tarik,<br></div>In newer version of matlab, using a "parfor" instead of a "for" can greatly speed up running ICA.<br></div><div><br></div>The code I'm posting here will run one instance of ICA (for different datasets) on each available core of the CPU, speeding up things almost linearly.<br><br>%%<br><br>INPath = 'Path to your data'; % Use something like this: fullfile(pwd, 'DATA', 'Ready_for_ICA');<br>OUTPath = 'Path where to save the datasets with ICA'; % Use something like this: fullfile(pwd, 'DATA', 'Done_ICA');<br><br>f = dir(fullfile(INPath,'*.set'));<br><br>%%<br><br>parfor s = 1:length(f)<br><br> [ALLEEG EEG CURRENTSET ALLCOM] = eeglab;<br> <br> EEG = pop_loadset('filename',f(s).name,'filepath',INPath);<br> <br> [ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0 );<br> <br> <br> EEG = pop_runica(EEG, 'extended',1,'interupt','on');<br> <br> [ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0 );<br> [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, 1,'savenew',fullfile(OUTPath, [f(s).name,'_ICA.set']),'overwrite','on','gui','off');<br><br>end<br><br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><span style="color:rgb(102,102,102)">Dr. Yamil Vidal<br></span></div><span style="color:rgb(102,102,102)">Cognitive Neuroscience Area<br>Scuola Internazionale Superiore di Studi Avanzati - SISSA<br>via Bonomea, 265<br>34136 Trieste, ITALY</span><br></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, Feb 15, 2018 at 9:00 PM, <span dir="ltr"><<a href="mailto:eeglablist-request@sccn.ucsd.edu" target="_blank">eeglablist-request@sccn.ucsd.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send eeglablist mailing list submissions to<br>
<a href="mailto:eeglablist@sccn.ucsd.edu">eeglablist@sccn.ucsd.edu</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://sccn.ucsd.edu/mailman/listinfo/eeglablist" rel="noreferrer" target="_blank">https://sccn.ucsd.edu/mailman/<wbr>listinfo/eeglablist</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:eeglablist-request@sccn.ucsd.edu">eeglablist-request@sccn.ucsd.<wbr>edu</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:eeglablist-owner@sccn.ucsd.edu">eeglablist-owner@sccn.ucsd.edu</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of eeglablist digest..."<br>
<br>Today's Topics:<br>
<br>
1. Re: About running ica on several machines (Tarik S Bel-Bahar)<br>
<br><br>---------- Forwarded message ----------<br>From: Tarik S Bel-Bahar <<a href="mailto:tarikbelbahar@gmail.com">tarikbelbahar@gmail.com</a>><br>To: Matthew Ma <<a href="mailto:matthewhang92@gmail.com">matthewhang92@gmail.com</a>><br>Cc: eeglablist <<a href="mailto:eeglablist@sccn.ucsd.edu">eeglablist@sccn.ucsd.edu</a>><br>Bcc: <br>Date: Wed, 14 Feb 2018 17:01:29 -0600<br>Subject: Re: [Eeglablist] About running ica on several machines<br><div dir="ltr"><div class="gmail_default" style="color:#333399">Hi Matthew, notes below. all the best.</div><div class="gmail_default" style="color:#333399">I am not sure but running a single ICA decomposition across several machines might be tricky or problematic. With a saavy programmer, you should be able to modify the code to, for example, have a different bias file name.</div><div class="gmail_default" style="color:#333399">I hope at least one of the suggestions below is useful to you. You will most likely get better answers from eeglab developers and other users who have solved issues similar to yours for their needs.</div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399">1. parallelized version of the code that can make the most use of parallelized matlab processing on a single computer</div><div class="gmail_default" style="color:#333399">2. explore cuda based ICA </div><div class="gmail_default" style="color:#333399">3. consider running multiple instances of matlab (on separate computers)</div><div class="gmail_default" style="color:#333399">4. use a gpu/processor bank/farm to send the "ica job" to</div><div class="gmail_default" style="color:#333399">5. saavy users can also likely benefit from cloud-based services, of course at some cost </div><div class="gmail_default" style="color:#333399">
<span style="color:rgb(51,51,153);font-family:arial,sans-serif;font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">some national programs that make free processors available to some researchers e.g., canada's sharknet I believe)</span>
<br></div><div class="gmail_default" style="color:#333399">6. check out EEGGIFT documentation for possible details regarding your question</div><div class="gmail_default" style="color:#333399">7. why not just run let each of your "machines" process one EEG file at a time. You are already ahead if you have "multiple machines" available :)</div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div><div class="gmail_default" style="color:#333399"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Feb 11, 2018 at 8:58 PM, Matthew Ma <span dir="ltr"><<a href="mailto:matthewhang92@gmail.com" target="_blank">matthewhang92@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear all,<div><br></div><div>I have been using eeglab for a while for preprocessing stuff, it is very useful so far. I have been noticing the processing speed of ica was quite low (I have already used binica and done downsampling of my data). I am thinking if I can make use of several machines which can be used to do the preprocessing.</div><div><br></div><div>As I noticed, the binica will generate the sphere and weight matrix as a file, also with the bias file. However, the bias file is always produced with the same file name, this gives rise to my question. As my work directory is located in another shared server(I access it using sshfs), I am wondering if the ica to produce wrong results if I run ica in several machines simutaneously (due to the same name of bias file...). I know the easiest solution is to duplicate the work directory to every local machine...but that seems a little bit stupid to do so.</div><div><br></div><div>Looking forward to your reply! Thank you very much.</div><div><br></div><div>Best regards,</div><div>Matthew</div></div>
<br>______________________________<wbr>_________________<br>
Eeglablist page: <a href="http://sccn.ucsd.edu/eeglab/eeglabmail.html" rel="noreferrer" target="_blank">http://sccn.ucsd.edu/eeglab/ee<wbr>glabmail.html</a><br>
To unsubscribe, send an empty email to <a href="mailto:eeglablist-unsubscribe@sccn.ucsd.edu" target="_blank">eeglablist-unsubscribe@sccn.uc<wbr>sd.edu</a><br>
For digest mode, send an email with the subject "set digest mime" to <a href="mailto:eeglablist-request@sccn.ucsd.edu" target="_blank">eeglablist-request@sccn.ucsd.e<wbr>du</a><br></blockquote></div><br></div>
<br>______________________________<wbr>_________________<br>
eeglablist mailing list <a href="mailto:eeglablist@sccn.ucsd.edu">eeglablist@sccn.ucsd.edu</a><br>
Eeglablist page: <a href="http://www.sccn.ucsd.edu/eeglab/eeglabmail.html" rel="noreferrer" target="_blank">http://www.sccn.ucsd.edu/<wbr>eeglab/eeglabmail.html</a><br>
To unsubscribe, send an empty email to <a href="mailto:eeglablist-unsub@sccn.ucsd.edu">eeglablist-unsub@sccn.ucsd.edu</a><br>
To switch to non-digest mode, send an empty email to <a href="mailto:eeglablist-nodigest@sccn.ucsd.edu">eeglablist-nodigest@sccn.ucsd.<wbr>edu</a><br></blockquote></div><br></div>