[Eeglablist] problem with file size

[Arnaud Delorme]

Dear Angel,

this option allows to perform most of the data processing on disk. 
Channels or trials are loaded and processed and one at a time.

Unlike SPM, EEGLAB was not designed with this type of disk processing in mind to start with. To be able to use all of the EEGLAB function that use passage of parameters by value, we had to find a way to pass the data by reference, something that is usually not possible in Matlab. This means that we had to implement some pretty dirty hacks (like using dbstack to look at the content of the stack and evaluating scripts in the caller function to check if the variable exist in the caller then copy it as a global temporary variable). We have a series of test (about 40) that check that the hack functions are doing what they are supposed to do. However, it is hard to guarantee that this implementation is bug free especially since it is not the default implementation and not heavily used by users which would help track potential bugs. Nevertheless this implementation passed the about 5000 EEGLAB independent test cases. It is quite unlikely that results of computation will be corrupted. Instead you can expect some rare functions will return an error while they should not.

Best,

Arno

On 11 May 2013, at 11:31, Angel Tabullo wrote:

> Arnaud:  I downloaded version 12 and it worked perfectly. May I ask what does this option do and what would be the potential risks?
> 
> De: Arnaud Delorme <arno at ucsd.edu>
> Para: mmiyakoshi at ucsd.edu 
> CC: Angel Tabullo <angeltabullo at yahoo.com>; "eeglablist at sccn.ucsd.edu" <eeglablist at sccn.ucsd.edu> 
> Enviado: Sábado, 11 de mayo, 2013 2:22 A.M.
> Asunto: Re: [Eeglablist] problem with file size
> 
> Dear Angel,
> 
> if you have EEGLAB 12, try using the option "If set, use memory mapped array..." in the "File > Memory and other option" menu item.
> 
> Best,
> 
> Arno
> 
> On 9 May 2013, at 17:26, Makoto Miyakoshi wrote:
> 
>> Dear Angel,
>> 
>> If you are using 32bit OS, even if you have 4GB RAM it may complain about memory. From my experience, importing ASCII somehow requires a lot of memory.
>> 
>> I would suggest you borrow 64bit OS machine to just import the data. Once it is imported and saved as a set + fdt file you'll probably able to load it with your 32bit OS machine.
>> 
>> Makoto 
>> 
>> 
>> 2013/5/2 Angel Tabullo <angeltabullo at yahoo.com>
>> Hi everyone! I'm trying to import raw EEG data in ASCII .txt file format to EEGLAB. EEG activity was recorded from 32 channels at 256 Hz and is aproximately 1:30 hour long, resulting in a 222.879 kb file. I was able to import it using EEGLAB version 7.9.2.20 (I tried the latest version first, but kept obtaining an "out of memory" error), and save it, but when I try to load the dataset I get the following message: "EEGLAB error in function eeg_getdataact() at line 162: Out of memory". Due to limitations in my acquisition software, I can't export the registry in other more compact formats like .EDF. 
>> 
>> I thought one possible solution would be to divide the file in different parts, filter and extract the epochs from each part, and then concatenate them all before ICA analysis, but I'm concerned that filtering each part separately and reuniting them later may not be an adequate procedure. What do you think? I would very much appreciate suggestions and comments.
>> 
>> Thank you!
>> 
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
>> 
>> 
>> 
>> -- 
>> Makoto Miyakoshi
>> Swartz Center for Computational Neuroscience
>> Institute for Neural Computation, University of California San Diego
>> _______________________________________________
>> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
>> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
>> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20130511/296154e9/attachment.html>