[Eeglablist] eeglablist Digest, Vol 160, Issue 7

[Yamil Vidal Dos Santos]

Re: About running ica on several machines (Tarik S Bel-Bahar)

Dear Tarik,
In newer version of matlab, using a "parfor" instead of a "for" can greatly
speed up running ICA.

The code I'm posting here will run one instance of ICA (for different
datasets) on each available core of the CPU, speeding up things almost
linearly.

%%

INPath         = 'Path to your data'; % Use something like this:
fullfile(pwd, 'DATA', 'Ready_for_ICA');
OUTPath        = 'Path where to save the datasets with ICA'; % Use
something like this: fullfile(pwd, 'DATA', 'Done_ICA');

f = dir(fullfile(INPath,'*.set'));

%%

parfor s = 1:length(f)

    [ALLEEG EEG CURRENTSET ALLCOM] = eeglab;

    EEG = pop_loadset('filename',f(s).name,'filepath',INPath);

    [ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0 );


    EEG = pop_runica(EEG, 'extended',1,'interupt','on');

    [ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0 );
    [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG,
1,'savenew',fullfile(OUTPath,
[f(s).name,'_ICA.set']),'overwrite','on','gui','off');

end


Dr. Yamil Vidal
Cognitive Neuroscience Area
Scuola Internazionale Superiore di Studi Avanzati - SISSA
via Bonomea, 265
34136 Trieste, ITALY

On Thu, Feb 15, 2018 at 9:00 PM, <eeglablist-request at sccn.ucsd.edu> wrote:

> Send eeglablist mailing list submissions to
>         eeglablist at sccn.ucsd.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://sccn.ucsd.edu/mailman/listinfo/eeglablist
> or, via email, send a message with subject or body 'help' to
>         eeglablist-request at sccn.ucsd.edu
>
> You can reach the person managing the list at
>         eeglablist-owner at sccn.ucsd.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of eeglablist digest..."
>
> Today's Topics:
>
>    1. Re: About running ica on several machines (Tarik S Bel-Bahar)
>
>
> ---------- Forwarded message ----------
> From: Tarik S Bel-Bahar <tarikbelbahar at gmail.com>
> To: Matthew Ma <matthewhang92 at gmail.com>
> Cc: eeglablist <eeglablist at sccn.ucsd.edu>
> Bcc:
> Date: Wed, 14 Feb 2018 17:01:29 -0600
> Subject: Re: [Eeglablist] About running ica on several machines
> Hi Matthew, notes below. all the best.
> I am not sure but running a single ICA decomposition across several
> machines might be tricky or problematic. With a saavy programmer, you
> should be able to modify the code to, for example, have a different bias
> file name.
> I hope at least one of the suggestions below is useful to you. You will
> most likely get better answers from eeglab developers and other users who
> have solved issues similar to yours for their needs.
>
> 1. parallelized version of the code that can make the most use of
> parallelized matlab processing on a single computer
> 2. explore cuda based ICA
> 3. consider running multiple instances of matlab (on separate computers)
> 4. use a gpu/processor bank/farm to send the "ica job" to
> 5. saavy users can also likely benefit from cloud-based services, of
> course at some cost
> some national programs that make free processors available to some
> researchers e.g., canada's sharknet I believe)
> 6. check out EEGGIFT documentation for possible details regarding your
> question
> 7. why not just run let each of your "machines" process one EEG file at a
> time. You are already ahead if you have "multiple machines" available :)
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sun, Feb 11, 2018 at 8:58 PM, Matthew Ma <matthewhang92 at gmail.com>
> wrote:
>
>> Dear all,
>>
>> I have been using eeglab for a while for preprocessing stuff, it is very
>> useful so far. I have been noticing the processing speed of ica was quite
>> low (I have already used binica and done downsampling of my data). I am
>> thinking if I can make use of several machines which can be used to do the
>> preprocessing.
>>
>> As I noticed, the binica will generate the sphere and weight matrix as a
>> file, also with the bias file. However, the bias file is always produced
>> with the same file name, this gives rise to my question. As my work
>> directory is located in another shared server(I access it using sshfs), I
>> am wondering if the ica to produce wrong results if I run ica in several
>> machines simutaneously (due to the same name of bias file...). I know the
>> easiest solution is to duplicate the work directory to every local
>> machine...but that seems a little bit stupid to do so.
>>
>> Looking forward to your reply! Thank you very much.
>>
>> Best regards,
>> Matthew
>>
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.uc
>> sd.edu
>> For digest mode, send an email with the subject "set digest mime" to
>> eeglablist-request at sccn.ucsd.edu
>>
>
>
> _______________________________________________
> eeglablist mailing list eeglablist at sccn.ucsd.edu
> Eeglablist page: http://www.sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsub at sccn.ucsd.edu
> To switch to non-digest mode, send an empty email to
> eeglablist-nodigest at sccn.ucsd.edu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20180216/ce3d4f95/attachment.html>