[Gc] RAM enough but out of virtual memory?

Boehm, Hans hans.boehm at hp.com
Mon Jul 18 14:25:55 PDT 2005


You are running out of virtual address space.  Most of the
allocated memory is never touched.  Thus the amount of
physical memory is largely irrelevant.  You could probably
get things to run slightly longer by using -DUSE_MMAP, which
might give you another 2GB of address space.  But that's not
the real problem here.

If you see lines like

Section 22 from 0xad4f000 to 0xb54f000 1520/2048 blacklisted
Section 23 from 0xb59f000 to 0xbd9f000 1554/2048 blacklisted
Section 24 from 0xbdef000 to 0xc5ef000 1568/2048 blacklisted

in the GC_DUMP_REGULARLY log, that means that more than 3/4
of newly allocated memory was unusable because the collector
found "pointers" to it before it was allocated, i.e. there
are known false references into it.  These numbers are
MUCH higher than normal.

Typically this can be easily avoided by changing a few allocations
of objects that are known to hold compressed data to
GC_MALLOC_ATOMIC, or telling the GC in some other way not
to scan such data.  If you're not sure where the collector is
finding the bogus pointers (and thus what it probably shouldn't
be scanning), you can get information by building the GC with
-DPRINT_BLACK_LIST.

Hans

> -----Original Message-----
> From: gc-bounces at napali.hpl.hp.com 
> [mailto:gc-bounces at napali.hpl.hp.com] On Behalf Of Zhang Le
> Sent: Monday, July 18, 2005 1:56 PM
> To: gc at napali.hpl.hp.com
> Subject: Re: [Gc] RAM enough but out of virtual memory?
> 
> 
> On 7/18/05, Boehm, Hans <hans.boehm at hp.com> wrote:
> > What's your platform?
> Linux 2.6 smp (2 PIII cpus) with 512 RAM, but threading is 
> disabled in GC.
> 
> > Can you run the program with the GC_PRINT_STATS and 
> GC_DUMP_REGULARLY 
> > environment variables set, and capture the resulting log?
> > 
> > That would tell us a lot more ...
> The log file is too big to paste here, please see: 
> http://homepages.inf.ed.ac.uk/s0450736/tmp/log.txt 
> (GC_PRINT_STATS, 20k) 
> http://homepages.inf.ed.ac.uk/s0450736/tmp/log2.txt
> (GC_PRINT_STATS+GC_DUMP_REGULARLY, 5.2M)
> 
> Interestingly, when I try to run my program on a Linux 
> machine with 4G RAM, it stops at the same position as on the 
> 512 RAM machine. I'd expected it runs longer!  Also, I keep 
> getting warning message like: GC Warning: Repeated allocation 
> of very large block (appr. size 20480):
>         May lead to memory leak and poor performance.
> after it consumes more than 400MB vram, though real RAM used 
> is always under 100MB reported by top.
> 
> Zhang Le
> 
> > 
> > It's possible that this can happen if the GC is scanning a lot of 
> > essentially random bits (e.g. compressed data).  But there's really 
> > too little information to go on, so far.
> > 
> > Hans
> > 
> > > -----Original Message-----
> > > From: gc-bounces at napali.hpl.hp.com 
> > > [mailto:gc-bounces at napali.hpl.hp.com] On Behalf Of Zhang Le
> > > Sent: Monday, July 18, 2005 9:00 AM
> > > To: gc at napali.hpl.hp.com
> > > Subject: [Gc] RAM enough but out of virtual memory?
> > >
> > >
> > > Hello,
> > >   I'm running my program with gc 6.5 on a machine with 512MB ram. 
> > > Then GC quits with a warning message : GC Warning: Out of 
> Memory!  
> > > Returning NIL!
> > >
> 
> _______________________________________________
> Gc mailing list
> Gc at linux.hpl.hp.com 
> http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
> 



More information about the Gc mailing list