[Eeglablist] importing edf, bdf - memory and speed issues

[Robert Oostenveld]

Dear Andre

The code snippet

> DATA=fread(fid,[3,numValues],'uint8');

will allocate matrix that can hold 3 x Numvalues double precision  
values. The values are between 0 and 255, but they are represented as  
double in matlab memory. That is because the fread function always  
returns doubles. The next code snippet

>     DATA=[1 256 65536]*DATA;

will start by allocating a temporary  1 x Numvalues matrix that can  
hold double precision, then it does the floating point matrix  
multiplication and the result is (value per value) copied into the  
1xNumvalues array. Once it is done with the multiplication, the  
temporary 1xNumvalues matrix will replace the 3xNumvalues matrix with  
the same name. So at the time of multiplication, your code indeed  
uses 4 times the amount of memory than required to store the final  
result, i.e. 4*8=32 bytes per sample. My c-code is very similar, but  
in C the buffer data is represented in unsigned chars, i.e. it only  
requires 3+8=11 bytes per sample. Furthermore, the integer  
multiplication (=byte shifting) that I use in my c-code is probably  
sligtly faster than the float multiplication that you do.

Your code will already be much faster than the original eeglab/biosig  
code, which reads 24 bits at a time and letx matlab do the  
conversion. But for BDF files of about 1GB which some people have to  
deal with, the 32-8=24 byte overhead per sample compared to the  
11-8=3 byte overhead in the mex file can make a large difference.

best
Robert