[Eeglablist] RAM usage when loading and saving (certain) EEGLAB files

[Armand Mensen]

Dear EEGLAB users,

I work with high-density sleep recordings at a moderate sampling rate of
500Hz. This leads to file sizes of around 16GB per night. My desktop has
64GB RAM, for this very reason.

However whenever I load particular set/fdt file (that were originally mff
files from EGI), it takes 4 times the RAM and then also starts using the
cache, slowing my computer down to a halt for several minutes. After
successful loading the RAM usage goes to the expected 16GB however the disk
cache grows for several minutes after loading is complete (my "swappiness"
value is low so disk cache should not be used if RAM is available.

The EEG structure appears completely normal once loading is complete. Yet,
I suspect that this still has something to do with the original file
structure, however implausible that seems since I have other large files I
work with that don't have this memory "leak" happening everytime they're
loaded.

1) Why does it need 4x the file size in RAM to load the file?
2) 2x the file sizes remain in the virtual disk cache

Any hints on how to debug this situation or hints as to what I could try to
actually work with these files would be greatly appreciated.

All the best,
Armand Mensen

*Postdoctoral Researcher*
*InselSpital Bern, Switzerland*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://sccn.ucsd.edu/pipermail/eeglablist/attachments/20170424/7d81389f/attachment.html>