[Eeglablist] [Fwd: Re: memory problems when reading in 128 channels Biosemi data]

[Toby Fernsler]

Nathaniel,
Good point about "pack", it was causing me problems with earlier Matlab 
versions and I had been avoiding it.  It seems to work ok now.
I was thinking fragmentation of the OS's memory pages would influence 
fragmentation of Matlab's memory space, but I don't really know.  
Anyone?  It may vary by OS as well...
As for Matlab64, Rey in our lab hit the ceiling at 60MB, but that might 
have been a hard drive space issue.
Thanks for the insightful reply,
Toby

-------- Original Message --------
Subject: 	Re: [Eeglablist] memory problems when reading in 128 channels 
Biosemi data
Date: 	Tue, 22 May 2007 21:51:57 -0700
From: 	Nathaniel Smith <njs at pobox.com>
To: 	Toby Fernsler <toby at sccn.ucsd.edu>

	


On Tue, May 22, 2007 at 03:18:54PM -0700, Toby Fernsler wrote:
>     One odd issue that Matlab has is memory fragmentation: any Matlab 
> variable must be loaded in a contiguous block of memory.  So even if you 
> have 2GB available, you may not be able to load a 1GB file if they're 
> all in chunks smaller than 1GB.  Matlab fragments its working RAM as it 
> works, and has no other means of addressing this than by restarting.  

Isn't this where the "pack" command comes in handy?

> Also it may not be able to grab excessively fragmented memory, which can 
> happen if the computer has been on for awhile.  So, as always, try 
> rebooting.

This seems wrong to me.  Each Matlab variable needs to occupy a
continuous span of Matlab's own virtual address space, but a
contiguous block of virtual memory does not have to be backed by a
continuous block of real memory -- the OS can and does assemble
physical memory pages in a pretty arbitrary fashion, and contiguous
blocks of physical memory are an _extremely_ rare commodity.

So the limiting factors for fragmentation issues aren't how long the
computer has been running or how much memory you have installed, but
rather how long Matlab has been running and how large your virtual
address space is -- if you run 32-bit Matlab for a long time you might
have problems, and if you're running 64-bit Matlab then in practice
fragmentation simply can't be an issue.

...If you're running on Windows then restarting might still help, of
course, just if it does it won't be for any reason accessible to human
understanding.

-- Nathaniel

-- 
Electrons find their paths in subtle ways.