[Eeglablist] script example, to extract epochs?

[Arnaud Delorme]

With respect to memory, we have implemented a scheme to use memory  
mapping (the data never gets loaded in memory). It proves to be  
unstable using some Matlab version (especially Linux 64-bit).

For each datasets, you must save two files, one for header (the set  
file) and one for the data (the .fdt file or .dat file). When you load  
such a dataset and have activated memory mapping in the option menu,  
virtually no memory is required (until you modify the dataset).

For Neuroscan users, Craig Rypstat and John Curtin implemented direct  
memory mapping at reading (so gigabyte files may be read under machine  
with low memory).

Computer savvy users may access the CVS EEGLAB repository at http://sccn.ucsd.edu/wiki/How_to_download_EEGLAB
Others will have to wait another few weeks until we extensively test  
the stability of new features and issue a new ZIP release.

Arno

On 25 juin 09, at 15:39, Hamish INNES-BROWN wrote:

> Memory always seems to be an issue.  I've heard using 64-bit matlab  
> will
> help, but that might not be possible (it's not for me at least), so  
> here
> is my solution (it's not pretty I admit): I try and do everything in
> small steps, but all the subjects at once, restarting matlab when it
> crashes and picking up where it left off... ie I do all the epoching  
> in
> one script (for all the subjects), then all the re-referencing in
> another, and so on.
>
> Matlab is bound to crash (memory), and the only solution seems to be  
> to
> actually restart matlab (the pack command doesn't do much, clear and
> restart eeglab doesn't work either). So I have the "input files" in  
> one
> directory ('rawpath' in the script below), then the output files go  
> into
> a separate directory ('savepath'). When matlab inevitabley runs out of
> memory on a bigger-than-usual .set file, it crashes, and I just look  
> in
> the 'savepath' directory and see where it got up to.  I then have a  
> 3rd
> directoy called "finished" - I move the sucessfully processed .set  
> files
> into the "finished" directory (usually just by hand in windows), then
> run the script again. The script works by finding a list of files in  
> the
> raw directory, so once the processed files are moved out of there,  
> it no
> longer sees them, and starts at the next un-processed file.
>
> So far I've been lucky, my files are just small enough that restarting
> matlab freshly will allow them to be epoched, but some of them wont
> epoch if anything else has been opened first. Seems the .set file  
> can be
> up to around 150MB for me.
>
> Like I said, it's not an elegant solution, but as you mentioned the
> memory issue only appeared on the 2nd time around the loop, I thought
> this might be the same problem.
>
> (this is for epoching continuous data, but I guess the principle is  
> the
> same)
>
>
> If anyone has a more elegant solution I'd also be glad to hear it.
>
>
>
>
> % Matlab script for epoching all the set files
>
> % 1) get list of set files in raw data directory
>
> rawpath =  'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\01raw\';
> savepath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\02epochs\';
> triggers = {'111' '122' '131' '141' '142' '151' '152' '162'};
>
> cd(rawpath);
> files = dir('*.set');
> nfiles = size(files,1);
>
>
> for i = 1:nfiles
>    %[ALLEEG EEG CURRENTSET ALLCOM] = eeglab;
>
>    EEG = pop_loadset( 'filename', files(i).name, 'filepath', rawpath);
>    EEG = eeg_checkset( EEG );
>    [ALLEEG EEG CURRENTSET ] = eeg_store(ALLEEG, EEG);
>    EEG = pop_epoch( EEG, triggers, [-.8 1], 'newname', [files(i).name
> 'epochs'], 'epochinfo', 'yes');
>    EEG = eeg_checkset( EEG );
>    EEG = pop_rmbase( EEG, [-200    0]);
>    EEG = eeg_checkset( EEG );
>    %EEG = pop_saveset( EEG,  'filename', strcat(files(i).name,
> '_epochs'), 'filepath', savepath);
>    [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, CURRENTSET,
> 'setname', 'epochs');
>    EEG = eeg_checkset( EEG );
>
>    EEG = pop_saveset( EEG,  'filename', strcat(files(i).name,
> '_epochs'), 'filepath', savepath);
>    ALLEEG = pop_delset( ALLEEG, [1 2] );
> end
>
>
> Hamish Innes-Brown
> Research Assistant, Music and Pitch Project
>
> The Bionic Ear Institute
> 384-388 Albert Street
> East Melbourne, VIC 3002
> Tel: +61 3 9667 7529
> Fax: +61 3 6997 7518
>
>
>
>
> ________________________________
>
> From: eeglablist-bounces at sccn.ucsd.edu
> [mailto:eeglablist-bounces at sccn.ucsd.edu] On Behalf Of Andrew Hill
> Sent: Thursday, 25 June 2009 5:50 PM
> To: eeglablist
> Subject: Re: [Eeglablist] script example, to extract epochs?
>
>
> several people have suggested solutions for this - thanks to everyone!
> i'm still having issues understanding a few things in EEGLab  
> scripting,
> and especially in memory management.
>
> here is my script:
>
>
>
>
>
> for S = [108 109];
>    eeglab
>    Cond =[ 'LVF_ConTarg_AllCue'; 'LVF_IncTarg_AllCue';
> 'RVF_ConTarg_AllCue'; 'RVF_IncTarg_AllCue' ];
>    trigs=[110 111 112 113; 120 121 122 123; 210 211 212 213; 220 221
> 222 223 ];
>
>
>
>
>   for c = 1:4;
>
>
>
>
>    %load a file, for example 108.set, which is not filtered and uses
> common vertex reference.
>    EEG = pop_loadset( 'filename', [int2str(S),'.set'], 'filepath',
> '/Users/andrew/Documents/Research/DATA/stage/');
>    [ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0 );
>    EEG = eeg_checkset( EEG );
>
>
>
>
>    % re-reference to common average reference, excluding VEOU and VEOL
>    EEG = pop_reref( EEG, [], 'refstate',0, 'exclude',[65 66] )
>
>
>
>
>    % lookup Channel Locations
>    EEG=pop_chanedit(EEG,  'lookup',
> '/Users/andrew/Documents/Research/eeglab2008October01_beta/plugins/ 
> dipfi
> t2.2/standard_BESA/standard-10-5-cap385.elp');
>
>
>
>
>    % EXTRACT EPOCHS, for trigs relevant to Cond
>    EEG = pop_selectevent( EEG, 'event',trigs(c,:) , 'deleteepochs',
> 'on');
>    [ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, 1, 'setname',
> Cond(c,:), 'savenew', [int2str(S),'_AVGREREF_',Cond(c,:)],  
> 'overwrite',
> 'on', 'gui', 'off');
>
>
>
>    %baseline filter using pre-stimulus interval.
>    %NOT working ->"pop_rmbase(): Wrong point range"
>    %EEG = pop_rmbase( EEG, [-1000  0]);
>
>
>
>
>    %maybe CLEAR some RAM? NOT working.
>    %ALLEEG = pop_delset( ALLEEG, [1] );
>    %CURRENTSET = pop_delset()
>    end;
> end;
>
> so some strangeness:
>
> 1) when i save a new .set named per the S(ubject) and Cond(ition), the
> 'deleteepochs', 'on'doesn't seem to work - the file contains ALL the
> epochs, not just 'trigs' ones specified./
>
>
> 2) the pop_rmbase syntax seems to be correct, from my eegh, but when  
> put
> in a script it throws "pop_rmbase(): Wrong point range" . and  
> including
> 'timerange' doesn't help.
>
>
> 3) the biggest problem is this script the second time around the loop,
> e.g. it only produces files for the first condition, first subject.
>
>
> 108_AVGREREF_LVF_ConTarg_AllCue.set
> 108_AVGREREF_LVF_ConTarg_AllCue.dat
>
> and then fails with out of memory errors:
> ??? Error using ==> ctranspose
> Out of memory. Type HELP MEMORY for your options.
>
> i tried ALLEEG = pop_delset( ALLEEG, [1] ); to clear memory before
> running through the Condition loop a 2nd time, but this doesn't work.
> what am i doing wrong? is there any better way to clear the memory
> before the next loop through?
>
>
> thanks,
> andrew
>
> On Jun 18, 2009, at 4:56 PM, Andrew Hill wrote:
>
>
>
>
> 	does anyone have a simple example script, to open a .set file
> and
> 	
>
> 	extract a series of epochs by event types into different files?
> 	
>
>
>
>
> _______________________________________________
> Eeglablist page: http://sccn.ucsd.edu/eeglab/eeglabmail.html
> To unsubscribe, send an empty email to eeglablist-unsubscribe at sccn.ucsd.edu
> For digest mode, send an email with the subject "set digest mime" to eeglablist-request at sccn.ucsd.edu