<html>
<body>
Dear Hamish,<br><br>
the problem you describe often occurs when Matlab has multiple set files
in memory at the same time. Preventing this is fairly simple, but not as
straightforward as one might expect.<br><br>
I'll use an example to explain this:<br><br>
EEG = pop_loadset( 'filename1', files(i).name, 'filepath', rawpath);
%Load first file<br>
% ...<br>
EEG = pop_loadset( 'filename2', files(i).name, 'filepath', rawpath);
%Load second file<br><br>
When the first line is executed, Matlab only needs memory to keep this
set file in memory. When the second line is executed, the first data-set
is still in memory, while the second dataset is being loaded. The first
set-file is only overwritten AFTER the second set-file is fully loaded,
i.e. Matlab requires memory enough to store both set-files.<br><br>
The following script is functionally the same, but does not have this
probem:<br><br>
EEG = pop_loadset( 'filename1', files(i).name, 'filepath', rawpath);
%Load first file<br>
% ...<br>
clear('EEG')<br>
EEG = pop_loadset( 'filename2', files(i).name, 'filepath', rawpath);
%Load second file<br><br>
Furthermore, when executing EEGLab in a Script, there is often no need to
keep a separate ALLEEG structure. I've edited your script to minimize
memory usage. This way, it should be able to run through all at once
without restarting:<br><br>
<blockquote type=cite class=cite cite="">% Matlab script for epoching all
the set files<br><br>
% 1) get list of set files in raw data directory<br><br>
rawpath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\01raw\';<br>
savepath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\02epochs\';<br>
triggers = {'111' '122' '131' '141' '142' '151' '152' '162'};<br><br>
cd(rawpath);<br>
files = dir('*.set');<br>
nfiles = size(files,1);<br><br>
<br>
for i = 1:nfiles<br>
EEG = pop_loadset( 'filename', files(i).name,
'filepath', rawpath);<br>
EEG = pop_epoch( EEG, triggers, [-.8 1], 'newname',
[files(i).name'epochs'], 'epochinfo', 'yes');<br>
EEG = pop_rmbase( EEG, [-200
0]);</blockquote>
EEG.setname='epochs';<br>
<blockquote type=cite class=cite cite=""> EEG =
pop_saveset( EEG, 'filename', strcat(files(i).name,'_epochs'),
'filepath', savepath);<br>
clear('EEG')<br>
end</blockquote><br><br>
I also removed the checkset commands to make things simpler. Those or
usually not necessary, unless there is some error in the
set-structure.<br><br>
best regards.<br>
Søren Andersen<br><br>
<br><br>
<br><br>
<blockquote type=cite class=cite cite="">% Matlab script for epoching all
the set files<br><br>
% 1) get list of set files in raw data directory<br><br>
rawpath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\01raw\';<br>
savepath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\02epochs\';<br>
triggers = {'111' '122' '131' '141' '142' '151' '152' '162'};<br><br>
cd(rawpath);<br>
files = dir('*.set');<br>
nfiles = size(files,1);<br><br>
<br>
for i = 1:nfiles<br>
%[ALLEEG EEG CURRENTSET ALLCOM] = eeglab;<br><br>
EEG = pop_loadset( 'filename', files(i).name,
'filepath', rawpath);<br>
EEG = eeg_checkset( EEG );<br>
[ALLEEG EEG CURRENTSET ] = eeg_store(ALLEEG,
EEG);<br>
EEG = pop_epoch( EEG, triggers, [-.8 1], 'newname',
[files(i).name<br>
'epochs'], 'epochinfo', 'yes');<br>
EEG = eeg_checkset( EEG );<br>
EEG = pop_rmbase( EEG, [-200
0]);<br>
EEG = eeg_checkset( EEG );<br>
%EEG = pop_saveset( EEG, 'filename',
strcat(files(i).name,<br>
'_epochs'), 'filepath', savepath);<br>
[ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG,
CURRENTSET,<br>
'setname', 'epochs');<br>
EEG = eeg_checkset( EEG );<br>
<br>
EEG = pop_saveset( EEG, 'filename',
strcat(files(i).name,<br>
'_epochs'), 'filepath', savepath);<br>
ALLEEG = pop_delset( ALLEEG, [1 2] );<br>
end</blockquote><br><br>
<br><br>
<br>
At 00:39 26.06.2009, you wrote:<br>
<blockquote type=cite class=cite cite="">Memory always seems to be an
issue. I've heard using 64-bit matlab will<br>
help, but that might not be possible (it's not for me at least), so
here<br>
is my solution (it's not pretty I admit): I try and do everything in<br>
small steps, but all the subjects at once, restarting matlab when it<br>
crashes and picking up where it left off... ie I do all the epoching
in<br>
one script (for all the subjects), then all the re-referencing in<br>
another, and so on. <br><br>
Matlab is bound to crash (memory), and the only solution seems to be
to<br>
actually restart matlab (the pack command doesn't do much, clear and<br>
restart eeglab doesn't work either). So I have the "input
files" in one<br>
directory ('rawpath' in the script below), then the output files go
into<br>
a separate directory ('savepath'). When matlab inevitabley runs out
of<br>
memory on a bigger-than-usual .set file, it crashes, and I just look
in<br>
the 'savepath' directory and see where it got up to. I then have a
3rd<br>
directoy called "finished" - I move the sucessfully processed
.set files<br>
into the "finished" directory (usually just by hand in
windows), then<br>
run the script again. The script works by finding a list of files in
the<br>
raw directory, so once the processed files are moved out of there, it
no<br>
longer sees them, and starts at the next un-processed file. <br><br>
So far I've been lucky, my files are just small enough that
restarting<br>
matlab freshly will allow them to be epoched, but some of them wont<br>
epoch if anything else has been opened first. Seems the .set file can
be<br>
up to around 150MB for me.<br><br>
Like I said, it's not an elegant solution, but as you mentioned the<br>
memory issue only appeared on the 2nd time around the loop, I
thought<br>
this might be the same problem. <br><br>
(this is for epoching continuous data, but I guess the principle is
the<br>
same)<br><br>
<br>
If anyone has a more elegant solution I'd also be glad to hear
it.<br><br>
<br><br>
<br>
% Matlab script for epoching all the set files<br><br>
% 1) get list of set files in raw data directory<br><br>
rawpath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\01raw\';<br>
savepath = 'D:\_PROJECTS\_FB\_ANALYSIS\__EEGLAB\SET\02epochs\';<br>
triggers = {'111' '122' '131' '141' '142' '151' '152' '162'};<br><br>
cd(rawpath);<br>
files = dir('*.set');<br>
nfiles = size(files,1);<br><br>
<br>
for i = 1:nfiles<br>
%[ALLEEG EEG CURRENTSET ALLCOM] = eeglab;<br><br>
EEG = pop_loadset( 'filename', files(i).name,
'filepath', rawpath);<br>
EEG = eeg_checkset( EEG );<br>
[ALLEEG EEG CURRENTSET ] = eeg_store(ALLEEG,
EEG);<br>
EEG = pop_epoch( EEG, triggers, [-.8 1], 'newname',
[files(i).name<br>
'epochs'], 'epochinfo', 'yes');<br>
EEG = eeg_checkset( EEG );<br>
EEG = pop_rmbase( EEG, [-200
0]);<br>
EEG = eeg_checkset( EEG );<br>
%EEG = pop_saveset( EEG, 'filename',
strcat(files(i).name,<br>
'_epochs'), 'filepath', savepath);<br>
[ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG,
CURRENTSET,<br>
'setname', 'epochs');<br>
EEG = eeg_checkset( EEG );<br>
<br>
EEG = pop_saveset( EEG, 'filename',
strcat(files(i).name,<br>
'_epochs'), 'filepath', savepath);<br>
ALLEEG = pop_delset( ALLEEG, [1 2] );<br>
end<br>
<br><br>
Hamish Innes-Brown<br>
Research Assistant, Music and Pitch Project<br><br>
The Bionic Ear Institute<br>
384-388 Albert Street<br>
East Melbourne, VIC 3002<br>
Tel: +61 3 9667 7529<br>
Fax: +61 3 6997 7518<br><br>
<br>
<br><br>
________________________________<br><br>
From: eeglablist-bounces@sccn.ucsd.edu<br>
[<a href="mailto:eeglablist-bounces@sccn.ucsd.edu" eudora="autourl">
mailto:eeglablist-bounces@sccn.ucsd.edu</a>] On Behalf Of Andrew
Hill<br>
Sent: Thursday, 25 June 2009 5:50 PM<br>
To: eeglablist<br>
Subject: Re: [Eeglablist] script example, to extract epochs?<br><br>
<br>
several people have suggested solutions for this - thanks to
everyone!<br>
i'm still having issues understanding a few things in EEGLab
scripting,<br>
and especially in memory management. <br><br>
here is my script:<br><br>
<br><br>
<br><br>
for S = [108 109];<br>
eeglab<br>
Cond =[ 'LVF_ConTarg_AllCue';
'LVF_IncTarg_AllCue';<br>
'RVF_ConTarg_AllCue'; 'RVF_IncTarg_AllCue' ];<br>
trigs=[110 111 112 113; 120 121 122 123; 210 211 212
213; 220 221<br>
222 223 ]; <br><br>
<br><br>
<br>
for c = 1:4;<br><br>
<br>
<br><br>
%load a file, for example 108.set, which is not
filtered and uses<br>
common vertex reference. <br>
EEG = pop_loadset( 'filename', [int2str(S),'.set'],
'filepath',<br>
'/Users/andrew/Documents/Research/DATA/stage/');<br>
[ALLEEG, EEG, CURRENTSET] = eeg_store( ALLEEG, EEG, 0
);<br>
EEG = eeg_checkset( EEG );<br><br>
<br>
<br><br>
% re-reference to common average reference, excluding
VEOU and VEOL<br>
EEG = pop_reref( EEG, [], 'refstate',0, 'exclude',[65
66] )<br><br>
<br><br>
<br>
% lookup Channel Locations<br>
EEG=pop_chanedit(EEG, 'lookup',<br>
'/Users/andrew/Documents/Research/eeglab2008October01_beta/plugins/dipfi<br>
t2.2/standard_BESA/standard-10-5-cap385.elp');<br><br>
<br><br>
<br>
% EXTRACT EPOCHS, for trigs relevant to Cond<br>
EEG = pop_selectevent( EEG, 'event',trigs(c,:) ,
'deleteepochs',<br>
'on');<br>
[ALLEEG EEG CURRENTSET] = pop_newset(ALLEEG, EEG, 1,
'setname',<br>
Cond(c,:), 'savenew', [int2str(S),'_AVGREREF_',Cond(c,:)],
'overwrite',<br>
'on', 'gui', 'off'); <br><br>
<br><br>
%baseline filter using pre-stimulus interval.
<br>
%NOT working ->"pop_rmbase(): Wrong point
range"<br>
%EEG = pop_rmbase( EEG, [-1000 0]);<br><br>
<br><br>
<br>
%maybe CLEAR some RAM? NOT working.<br>
%ALLEEG = pop_delset( ALLEEG, [1] );<br>
%CURRENTSET = pop_delset()<br>
end;<br>
end;<br><br>
so some strangeness:<br><br>
1) when i save a new .set named per the S(ubject) and Cond(ition),
the<br>
'deleteepochs', 'on'doesn't seem to work - the file contains ALL the<br>
epochs, not just 'trigs' ones specified./<br><br>
<br>
2) the pop_rmbase syntax seems to be correct, from my eegh, but when
put<br>
in a script it throws "pop_rmbase(): Wrong point range" . and
including<br>
'timerange' doesn't help.<br><br>
<br>
3) the biggest problem is this script the second time around the
loop,<br>
e.g. it only produces files for the first condition, first
subject.<br><br>
<br>
108_AVGREREF_LVF_ConTarg_AllCue.set<br>
108_AVGREREF_LVF_ConTarg_AllCue.dat<br><br>
and then fails with out of memory errors:<br>
??? Error using ==> ctranspose <br>
Out of memory. Type HELP MEMORY for your options.<br><br>
i tried ALLEEG = pop_delset( ALLEEG, [1] ); to clear memory before<br>
running through the Condition loop a 2nd time, but this doesn't
work.<br>
what am i doing wrong? is there any better way to clear the memory<br>
before the next loop through?<br><br>
<br>
thanks,<br>
andrew<br><br>
On Jun 18, 2009, at 4:56 PM, Andrew Hill wrote:<br><br>
<br>
<br><br>
<x-tab> </x-tab>does
anyone have a simple example script, to open a .set file<br>
and<br>
<x-tab> </x-tab><br><br>
<x-tab> </x-tab>extract a
series of epochs by event types into different files?<br>
<x-tab> </x-tab><br><br>
<br><br>
<br>
_______________________________________________<br>
Eeglablist page:
<a href="http://sccn.ucsd.edu/eeglab/eeglabmail.html" eudora="autourl">
http://sccn.ucsd.edu/eeglab/eeglabmail.html</a><br>
To unsubscribe, send an empty email to
eeglablist-unsubscribe@sccn.ucsd.edu<br>
For digest mode, send an email with the subject "set digest
mime" to eeglablist-request@sccn.ucsd.edu</blockquote>
<x-sigsep><p></x-sigsep>
Dr. Søren K. Andersen<br><br>
<i>Universität Leipzig <br>
Institut Psychologie I<br>
Allgemeine Psychologie & Methodenlehre</i> <br>
<i>Seeburgstr. 14-20</i> <br>
<i>D-04103 Leipzig</i> <br>
<i>Tel.: +49 - 341 - 97 39 54 4<br>
Fax: +49 - 341 - 97 35 96 9</i> <br><br>
E-Mail:
<a href="mailto:andersen@rz.uni-leipzig.de">andersen@rz.uni-leipzig.de</a>
<br>
<a href="http://www.uni-leipzig.de/~psyall2/andersen/index.html" eudora="autourl">
http://www.uni-leipzig.de/~psyall2/andersen/index.html</a> </body>
</html>