[Eeglablist] Memory and MATLAB "pack" command

[Tommaso Fadini]

Dear Arno and Andreas,
thank you for your answers and hints.

I firstly thought to manage the files simply implementing my swap  
space under linux, but it seems to me to be a non sense (the problem  
of managing Gb of data remains).

I like very much the proposed solutions number 2 and 3 (see below)  
and I am going to try them to find out which suits best my needs.


Optimizing the reading routines would be an exciting challenge...


By the way, I got yesterday with eeglab6.01b a (one time)  
segmentation error (see https://sccn.ucsd.edu/eeglab/bugzilla/ 
show_bug.cgi?id=59    I haven't used the patch and the error didn't  
show a second time).


Greetings
Tommaso



On May 9, 2007, at 7:35 PM, arno wrote:

> Dear Tommaso,
>
> using the "pack" command will not help much. "pack" simply  
> defragment Matlab memory but does not increase it. Usually, not  
> being able to read data has two causes: your recording sampling  
> rate is high and the files are "very" big or the loading routines  
> are not optimized enough and use too much memory. Here are some  
> workaround to read your files.
> 1) close all existing programs before running Matlab or buy more  
> memory (Matlab supports up to 16 Gb of Ram under 64-bit opterons).
>
> 2) read portion of continuous or epoch files, resample them, then  
> concatenate them using menu "Edit > Merge datasets". We will update  
> the merge command to be able to merge files "on disk" and delete  
> the existing datasets in memory. This way, it is sure that you will  
> have enough memory to merge any number of datasets (although you  
> might not have enough memory to reload the merged file!).
>
> 3) read a few channels at a time, then resample your data, and when  
> you are done fuse the files. For instance after reading the first  
> channels into dataset 1 and the rest into dataset 2, to fuse them  
> on the command line using type:
>
> EEG = ALLEEG(2);
> EEG.data(end+1: end+ALLEEG(1).nchan,:) = ALLEEG(1).data;
> if ~isempty(EEG.chanlocs)
>    EEG.chanlocs(end+1: end+ALLEEG(1).nchan) = ALLEEG(1).chanlocs;
> end;
> EEG = eeg_checkset(EEG);
> [ALLEEG EEG CURRENTSET] = eeg_store(ALLEEG, EEG);
> eeglab redraw;
>
> 4) optimize the reading routines to minimize memory usage (and send  
> them back to us please!).
>
> Arno
>
> Tommaso Fadini wrote:
>> Hello to all eeglablist members,
>> I write to you about the problem which some of you have already   
>> posted: not enough memory to load files.
>>
>> I would like to ask if someone knows if it is possible to use the   
>> "pack" command (or some other one while running EEGLAB) with  
>> EEGLAB  6.01b to write the loaded file to disk under linux  
>> (http:// www.mathworks.com/support/tech-notes/1100/1106.html).
>>
>> Using Matlab, the swap space under linux is managed automatically  
>> or  there is a way to handle it?
>>
>