[Eeglablist] Parallel Data Processing

[James Desjardins]

Hi Richard,

It sounds like you are interested in serial farming matlab scripts that use EEGLAB functions, where each worker will be doing something completely independent (e.g. load at file, then run ica, then save a file). So they can be run at the same time (or at different times, it does not really matter).

So with a parfor loop you would do something like:

parfor i=1:lenght(filenamelist)
    load in filename i
    run ica
    save out filename i
end

I have done this kind of thing with EEGLAB and Matlab PCT (version R2010b, on SHARCNet Linux clusters). EEGLAB does not have a problem with this.

If this is the kind of serial farming that you are interested in I recommend trying to run all of your serial scripts in Octave. You can run as many instances of Octave as you want on a cluster (until the hardware is saturated) and EEGLAB functions are very compatible with Octave execution (EEGLAB functions run in Octave with no modifications if you call octave with the --traditional option... also the Octave equivalents for the Signal and Statistics toolboxes are readily available).

James Desjardins, MA
Electrophysiology Technologist
Cognitive and Affective Neuroscience Lab, Psychology Department
Jack and Nora Walker Centre for Lifespan Development Research
High Performance Computing Consultant
SHARCNet, Compute/Calcul Canada

Brock University
500 Glenridge Ave.
St. Catharines, ON, Canada L2S 3A1
905-688-5550 x4676
--
"'Cause you never can tell What goes on down below!
"This pool might be bigger Than you or I know!"

McElligot's Pool
Dr.Seuss 1947

________________________________________
From: eeglablist-bounces at sccn.ucsd.edu [eeglablist-bounces at sccn.ucsd.edu] on behalf of Richard Höchenberger [richard.hoechenberger at gmail.com]
Sent: November-03-14 5:27 AM
To: eeglablist at sccn.ucsd.edu
Subject: [Eeglablist] Parallel Data Processing

Hello! We are planning to set up a server for running our EEG analyses
with EEGLAB. The Parallel Computing Toolbox (PCT) [1] in conjunction
with the Distributed Computing Server (DCS) [2] appears to be pretty
promising to me.

The PCT allows for the straightforward execution of calculations in a
parallel manner. In the most simple approach, 'for' loops could just
be replaced with 'parfor' loops, which would execute the loop
iterations independently on different workers.

Now I do understand that this approach only works if one iteration
does not depend on the results of another (because they are processed
in parallel). However, I was wondering

o if EEGLAB keeps internal states (caches, heuristics, ...) during a
calculation, which might then have implications on a second
calculation running in parallel, and if so,
o whether the worker processes are actual separate Matlab processes
completely independent of one another, meaning internal EEGLAB states
in one worker would not influence calculations in another. (I could
not find any documentation about this particular question on the
Matlab website.)

Thank you very much for your help!

Richard


References:
[1] http://www.mathworks.com/products/parallel-computing/index.html
[2] http://www.mathworks.com/products/distriben/index.html
_______________________________________________
Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20141104/456a129b/attachment.html>