[Eeglablist] EEGlab parallel computing & hardware specs

[Andreas Widmann]

Hi Avi,

short answer:
upgrading to a recent 12.x EEGLAB version and using pop_eegfiltnew should solve your problem.

Long answer:
* I have to slightly correct my previous statement. The legacy (fir1/firls) FIR filter uses MATLAB filtfilt instead of MATAB filter and is thus not multi-threaded by default.
* The legacy filter is flawed (see e.g. http://www.frontiersin.org/Perception_Science/10.3389/fpsyg.2012.00233/full and https://sccn.ucsd.edu/bugzilla/show_bug.cgi?id=631for a more detailed description) and is not recommended.
* The new FIR filter in EEGLAB 12.x uses plain MATLAB filter an is thus genuinely multi-threaded. Filtering a 1086734-point 66-channel dataset takes 5 seconds with 265-point filter (default for 100 Hz-lowpass) and 2 min 16 seconds with 6145-point filter (your filter length) on my 1.7 Ghz dual core i5. Should be much faster on your machine.
* Note: there is an inconsistency between your command (EEG = pop_eegfilt( EEG, 0, 100, [], 0,0, 0, ‘firls’, 0);) and your reported output. This command should give a 60-point lowpass filter and *not* a 6144-point highpass. Did you change something in your EEGLAB code?

Hope this helps! Best,
Andreas

> I am using EEGlab to process EEG recordings, and running low pass 100 Hz is very slow. I am looking for code modifications I can make to accelerate the running time, and also recommendation for optimal hardware specs for this process.
> 
> The files I process are recordings of 66 channles @ 2KHz ~10 minutes long (1086734 samples) bug. I can’t resample the files because I need high temporal resolution.
> 
> I use EEGlab version 11.0.3.1b, and run “firls” filter. I run the filter with the following command:
> EEG = pop_eegfilt( EEG, 0, 100, [], 0,0, 0, ‘firls’, 0);
> When I run this command I get the following message:
> 
> eegfilt() - performing 6144-point highpass filtering.
> eegfilt() - highpass transition band width is 0.2 Hz.
>  
> The computer I run the processing on now is i5 3470 (4 cores@ 3.2 Ghz) with 4Gb RAM running windows 7 enterprise 64 bit and Matlab 2012a.
> 
> I tried running the same process on an i7 extreme edition 3930k with 64Gb RAM system and it didn't shorten the running time at all. I also tried running this filter on a workstation with Xeon CPU and it shortened the running time, but not significantly. I looked on the CPU monitor and found that it doesn’t use more than 1 core, and that the maximal RAM memory usage during the processing was about 6.5 Gb.
> 
> 1.     I found that using SSD shortens the time significantly, and therefore I assume that one of the big bottlenecks of this processing is that the process is made on the hard disk and not on the RAM. Is there a way to “make” Matlab use RAM instead of the hard drive?
> 
> 2.     Are there code changes I can make in EEGlab’s code to run this filter multithreaded so that the process will use all the cores of my CPU? Is there any way to make this process on GPU? Will it accelerate the process more than using parallel computing of the CPU? 
> 
>  
> Thank you
> 
> Avi
> 
> 
> 
> 2013/6/8 Arnaud Delorme <arno at sccn.ucsd.edu>
> Thanks Andreas for the clarification.
> 
> Arno
> 
> On 8 Jun 2013, at 06:58, Andreas Widmann wrote:
> 
> > Hi Avi and Arno,
> >
> > no using parfor won't help here, filtering is already multithreaded since MATLAB R2007a.
> > http://www.mathworks.de/support/solutions/en/data/1-4PG4AN/
> >
> > I can try to narrow down the problem, but please on list (eeglablist). This might be interesting also for others. Many relevant parameters are missing:
> > * Length of dataset (samples or time and fs)
> > * Filter length or order
> > * OS and version
> > * EEGLAB version
> > * Why legacy filter (fir1/firls)? Using the new filter will immediately halve computation time.
> >
> > Best,
> > Andreas
> >
> > Am 07.06.2013 um 22:20 schrieb Arnaud Delorme <arno at sccn.ucsd.edu>:
> >
> >> Dear Avi,
> >>
> >> yes, it is possible to shorten the time.
> >> If you have multiple processor, you could try replacing line 74 of firfilt.m
> >>
> >> for iDc = 1:(length(dcArray) - 1)
> >>
> >> with
> >>
> >> parfor iDc = 1:(length(dcArray) - 1)
> >>
> >>
> >> Channels will be filtered in parallel.
> >> GPU could help as well, but that's another story.
> >>
> >> Hope this helps,
> >>
> >> Arno
> >>
> >> On 2 Jun 2013, at 23:53, Avi Lazarovits wrote:
> >>
> >>> Hello,
> >>> My name is Avi and I work in lab with EEG recordings. I am using the EEGlab on matlab 2012b and would like to thank you for writing this useful toolbox.
> >>>
> >>> the Processing time of the fir1/firls lowpass filters is very long (20 minutes for 66 channels SSD and I am looking for ways to shorten it. Is there any way to shorten it by changing the code for using parallel computingqGPU computing? will EEGlab use parallel computing in next versions?
> >>> If not, what are the best hardware specs to run these filters? I tried running it on an i7 3930k with 64Gb RAM system and it didn't shorten the time vs. my i5 3470 4Gb RAM system. I also tried to run this filter on a workstation system with Xeon CPU and it shortened the running time, but not significantly. I found that using SSD shortens the time significantly. Do you have any other hardware specs recommendations?
> >>>
> >>> Thank you
> >>> Avi
> >>
> >
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4602 bytes
Desc: not available
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20130609/e74345b2/attachment.p7s>