[Eeglablist] How to use parallel pools in the std_precomp function?

Tue Jun 9 03:01:55 PDT 2015

Hi Katharina,

The break is for when you specify ‘cell’ parameter when calling std_precomp.m. This was implemented for debugging purposes, so removing the line with the break is fine. 
There is no reason for returning different number of frequencies. My guess is that something did change. The parfor loop is very straightforward. It is just a loop performed in parallel. It will not change anything else.

Best,

Arno

> On Jun 8, 2015, at 11:57 PM, Katharina Ledergerber <k.ledergerber at psychologie.uzh.ch> wrote:
> 
> Dear Arno,
> 
> thanks a lot. The parallel pools work and computations is much faster.
> However, I have now the problem that results are slightly different from what I calculated without parallel pools. I changed the line you told me. And the section looks like this:
> 
> parfor index = 1:length(STUDY.design(g.design).cell)
>             if ~isempty(g.cell)
>                  desset = STUDY.design(g.design).cell(g.cell);
>             else desset = STUDY.design(g.design).cell(index);
>             end;
>             if strcmpi(computewhat, 'channels')
>                 [tmpchanlist opts] = getchansandopts(STUDY, ALLEEG, chanlist, desset.dataset, g);
>                 std_ersp(ALLEEG(desset.dataset), 'channels', tmpchanlist, 'type', type, 'fileout', desset.filebase, 'trialindices', desset.trials, opts{:}, tmpparams{:});
>             else
>                 if length(desset.dataset)>1 && ~isequal(chanlist{desset.dataset})
>                     error(['ICA decompositions must be identical if' 10 'several datasets are concatenated to build' 10 'the design, abording' ]);
>                 end;
>                 std_ersp(ALLEEG(desset.dataset), 'components', chanlist{desset.dataset(1)}, 'type', type, 'fileout', desset.filebase, 'trialindices', desset.trials, tmpparams{:});
>             end;
>             if ~isempty(g.cell), break; end;
>         end;
> 
> The problem here is, that there is a break argument in the second last line, which is not valid for parfor loops (produces huge error messages). What does the  if ~isempty(g.cell), break; end; exactly do? 
> To make the function work I removed this line including the break argument. Could this be the reason for why I have different results? I also noticed that before I used the parfor loop, the precomputation gave me 102 ersptimes and 40 erspfreqs. With the parfor loop I now obtain still 102 ersptime, but also 102 erspfreqs. Why do I suddenly have a higher frequency resolution? Apart from canceling the line with the break argument, I did not change anything at all.
> 
> Thanks in advance for your help.
> Best,
> Katharina
> 
> 
> Katharina Ledergerber, MSc.
> PhD student
> University of Zurich
> Developmental Psychology:
> Childhood and Infancy 
> Binzmuehlestrasse 14, Box 21
> CH-8050 Zurich
> 
> Am 08.06.2015 um 08:00 schrieb Arnaud Delorme <arno at ucsd.edu <mailto:arno at ucsd.edu>>:
> 
>> Dear Katharina,
>> 
>> I think the standard way to change line 483 of std_precomp (in my version) from 
>> 
>>         for index = 1:length(STUDY.design(g.design).cell)
>> 
>> to
>> 
>>         parfor index = 1:length(STUDY.design(g.design).cell)
>> 
>> Then type in on the command line 
>> 
>> matlabpool(4);
>> 
>> Then run the spectral computation. It would make sense to change this line by default since parfor defaults to for by default (so we will do it).
>> Best,
>> 
>> Arno
>> 
>>> On Jun 7, 2015, at 5:18 PM, Makoto Miyakoshi <mmiyakoshi at ucsd.edu <mailto:mmiyakoshi at ucsd.edu>> wrote:
>>> 
>>> Dear Arno and Ramon,
>>> 
>>> Please help me with this question about the use of parfor().
>>> 
>>> Makoto
>>> 
>>> On Thu, Jun 4, 2015 at 6:48 AM, Katharina Ledergerber <k.ledergerber at psychologie.uzh.ch <mailto:k.ledergerber at psychologie.uzh.ch>> wrote:
>>> Hi everyone,
>>> 
>>> I am quite new to EEGLAB and I hope that I can explain my problem properly. I have a question on how to use parallel pools with the std_precomp function.
>>> 
>>> Currently I am applying time-frequency analyses on my high-density EEG data, recorded with a 128 electrode system. Therefore I use the standard std_precomp function to precompute channel measure ERSP/ITC on my files in the STUDY (for all subjects and all channels):
>>> 
>>> [STUDY ALLEEG] = std_precomp(STUDY, ALLEEG, {}, 'design', 1, 'ersp', 'on', 'itc', 'on');
>>> STUDY = pop_savestudy( STUDY, ALLEEG, 'filename', STUDY.filename, 'filepath', STUDY.filepath, 'savemode', 'resave');
>>> 
>>> As it is an FFT, it takes quite a while to compute (right now approx. 100 min per subject). Since I do not have a very powerful computer (4 cores, 8GB RAM), I thought I could try the precomputation on a more powerful computer (16 cores, 32GB RAM) to save time. To make use of all the cores I tried to apply parpool.
>>> 
>>> parpool (‚local‘, 8)
>>> [STUDY ALLEEG] = std_precomp(STUDY, ALLEEG,{} , 'design', 1, 'ersp', 'on', 'itc', 'on');
>>> STUDY = pop_savestudy( STUDY, ALLEEG, 'filename', STUDY.filename, 'filepath', STUDY.filepath, 'savemode', 'resave‘);
>>> delete(gcp);
>>> 
>>> It seemed to work, since all 16 cores showed activity. Strangely, the more powerful computer took as much time as my computer. Could it be that jobs are not distributed among cores, but all cores do exactly the same? Since I am very new to EEGLAB I have still some trouble understanding what the std_precomp function does in detail. I guess to distribute jobs among cores I should implement parfor loops in the std_precomp function instead of the ordinary for loops? I think, though that it is not as easy to implement, since parfor loops do have some restrictions in terms of what you can put in the loop.
>>> Does anyone have a function already modified with parfor loops? Otherwise, I am grateful for any advice about using parallel pools and ERSP precomputation.
>>> 
>>> Thanks,
>>> Katharina
>>> 
>>> Katharina Ledergerber, MSc.
>>> PhD student
>>> University of Zurich
>>> Developmental Psychology:
>>> Childhood and Infancy
>>> Binzmuehlestrasse 14, Box 21
>>> CH-8050 Zurich
>>> _______________________________________________
>>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html <http://sccn.ucsd.edu/eeglab/eeglabmail.html>
>>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu <mailto:eeglablist-unsubscribe at sccn.ucsd.edu>
>>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu <mailto:eeglablist-request at sccn.ucsd.edu>
>>> 
>>> 
>>> 
>>> -- 
>>> Makoto Miyakoshi
>>> Swartz Center for Computational Neuroscience
>>> Institute for Neural Computation, University of California San Diego
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20150609/2ba554ce/attachment.html>