[Gc] Out of Memory! Returning NIL!

Boehm, Hans hans.boehm at hp.com
Tue Nov 14 17:58:40 PST 2006


The log (not posted due to size) starts to look very problematic around
here:

--> Marking for collection 6 after 0 allocd bytes + 0 wasted bytes
Collection 5 reclaimed 0 bytes ---> heapsize = 1458176 bytes
World-stopped marking took 16 msecs
Bytes recovered before sweep - f.l. count = 0
Immediately reclaimed 0 bytes in heap of size 1458176 bytes(0 unmapped)
0 (atomic) + 112396 (composite) collectable bytes in use
Finalize + initiate sweep took 0 + 0 msecs
Complete collection took 16 msecs
Initiating full world-stop collection 7 after 0 allocd bytes
0 bytes in heap blacklisted for interior pointers
--> Marking for collection 7 after 0 allocd bytes + 0 wasted bytes
Collection 6 reclaimed 0 bytes ---> heapsize = 1458176 bytes
World-stopped marking took 15 msecs
Bytes recovered before sweep - f.l. count = 0
Immediately reclaimed 0 bytes in heap of size 1458176 bytes(0 unmapped)
0 (atomic) + 112396 (composite) collectable bytes in use
Finalize + initiate sweep took 0 + 0 msecs
Complete collection took 31 msecs
Initiating full world-stop collection 8 after 0 allocd bytes
0 bytes in heap blacklisted for interior pointers
--> Marking for collection 8 after 0 allocd bytes + 0 wasted bytes
Collection 7 reclaimed 0 bytes ---> heapsize = 1458176 bytes
World-stopped marking took 31 msecs
Bytes recovered before sweep - f.l. count = 0
ImmediatIncreasing heap size by 11304960 after 44256456 allocated bytes
Initiating full world-stop collection 7 after 46145016 allocd bytes
8192 bytes in heap blacklisted for interior pointers
--> Marking for collection 7 after 46145016 allocd bytes + 4291876
wasted bytes
Found new system malloc AllocationBase at 0x6a00000
GC Warning: Thread stack pointer 0x5e5fe60 out of range, pushing
everything
Collection 6 reclaimed 44248 bytes ---> heapsize = 45211648 bytes
World-stopped marking took 235 msecs
Bytes recovered before sweep - f.l. count = -151092
Immediately reclaimed -151092 bytes in heap of size 45211648 bytes(0
unmapped)
0 (atomic) + 3415544 (composite) collectable bytes in use
Finalize + initiate sweep took 0 + 0 msecs
Complete collection took 235 msecs
Initiating full world-stop collection 8 after 39636 allocd bytes
8241152 bytes in heap blacklisted for interior pointers
--> Marking for collection 8 after 39636 allocd bytes + 4092 wasted
bytes
GC Warning: Thread stack pointer 0x5e5fe60 out of range, pushing
everything
Collection 7 reclaimed -132308 bytes ---> heapsize = 45211648 bytes
World-stopped marking took 141 msecs
Bytes recovered before sweep - f.l. count = -21392
Immediately reclaimed -21392 bytes in heap of size 45211648 bytes(0
unmapped)
0 (atomic) + 3474584 (composite) collectable bytes in use
Finalize + initiate sweep took 0 + 0 msecs
Complete collection took 141 msecs
Initiating full world-stop collection 9 after 21712 allocd bytes
8241152 bytes in heap blacklisted for interior pointers

The collection numbers are not monotonically increasing, the heap size
suddenly jumps without indication of heap expansion, log entries are
interleaved, eventhough I think they should all be protected by the
allocation lock, and warning messages about the thread stack pointer
start appearing.  I have no idea what's going on, but it's very strange.
Did you somehow get two copies of the GC linked in?  Was there some
other opportunity for the log to get garbled?

It may be easiest to debug this by either setting a break point where
the "Thread stack pointer out of range" is generated, and seeing why the
collector sees a bad stack pointer, or by trying to understand why
GC_gc_no seems to go backwards.

Hans


________________________________

	From: Simon Tsai [mailto:mtsai at adobe.com] 
	Sent: Tuesday, November 14, 2006 4:16 PM
	To: Boehm, Hans
	Cc: gc at napali.hpl.hp.com
	Subject: RE: [Gc] Out of Memory! Returning NIL!
	
	

	I have tried to enable "USE_MUNMAP" and "USE_MMAP". I saw gc lib
called VirtualFree() under GC_unmap. I ran my testing program. The "VM
Size" is 1.3 GByte and "Mem Usage" is 221k under Task Manager. I have
stop my testing program to avoid "too many heap section". 

	Please review the attached gc.log. 

	Any suggestions,

	 

	Thanks,

	 

	simon

	 

	________________________________

		From: gc-bounces at napali.hpl.hp.com
[mailto:gc-bounces at napali.hpl.hp.com] On Behalf Of Boehm, Hans
	Sent: Tuesday, November 14, 2006 2:59 PM
	To: Simon Tsai
	Cc: gc at napali.hpl.hp.com
	Subject: RE: [Gc] Out of Memory! Returning NIL!

	 

	That's probably normal.  The collector in its default
configuration does not return memory to the OS; it only makes it
available to future allocations.  If you allocate another 800 MB, you
should not see much additional growth.

	 

	Hans

		 

		________________________________

				From: Simon Tsai
[mailto:mtsai at adobe.com] 
		Sent: Tuesday, November 14, 2006 1:53 PM
		To: Boehm, Hans
		Cc: gc at napali.hpl.hp.com
		Subject: RE: [Gc] Out of Memory! Returning NIL!

		I checked the gc library (version 6.8) testing program
test.c. 

		I have written a test program under Windows, it calls
GC_INIT(), then it allocates 800 Mbyte - 1 GByte memory - each time I
allocate 10k memory. I saw the VM Size is 1.2 GByte memory, even after I
release (not free) all memory pointer and called the following
functions:

		 

		                  while (GC_collect_a_little()) { }

		                    GC_gcollect();

		                    GC_invoke_finalizers();

		                  

		 

		Do I miss anything else? 

		 

		Thanks,

		 

		simon

		 

		________________________________

				From: gc-bounces at napali.hpl.hp.com
[mailto:gc-bounces at napali.hpl.hp.com] On Behalf Of Boehm, Hans
		Sent: Tuesday, November 14, 2006 11:56 AM
		To: Simon Tsai
		Cc: gc at napali.hpl.hp.com
		Subject: RE: [Gc] Out of Memory! Returning NIL!

		 

		That happens after the heap is already quite large, and
it is controlled by MAXHINCR defined in gc_priv.h.  It's also expected
behavior, since the collector would rather allocate the heap in fewer,
larger segments.  Normally the only cost is that you may end up with up
to 16MB of allocated address space per (large) process that isn't really
being touched.

		 

		The question is really why GC_expend_hp_inner() is
called so many times.

		 

		Hans

			 

			________________________________

						From: Simon Tsai
[mailto:mtsai at adobe.com] 
			Sent: Tuesday, November 14, 2006 11:00 AM
			To: Boehm, Hans
			Cc: gc at napali.hpl.hp.com; gc at napali.hpl.hp.com
			Subject: RE: [Gc] Out of Memory! Returning NIL!

			After debug, I found the following
information's:

			 

			GC_collect_or_expand(needed_blocks,
ignore_off_page)

			{

			    blocks_to_get = 4096;

			}

			 

			GC_expand_hp_inner(n)

			{

			n=4096;

			bytes = n * HBLKSIZE; 

			    bytes = 4096 * 4096 = 16777216;

			 

			}

			 

			 

			Why blocks_to_get grow so big? Is any way to
reduce this number? From Task manager, I saw VM size grow to 1.9 GByte. 

			 

			Thanks,

			 

			Simon

			 

			________________________________

						From: Boehm, Hans
[mailto:hans.boehm at hp.com] 
			Sent: Monday, November 13, 2006 5:53 PM
			To: Simon Tsai
			Cc: gc at napali.hpl.hp.com
			Subject: RE: [Gc] Out of Memory! Returning NIL!

			 

			It would be useful to look at a more complete GC
log (either define the GC_PRINT_STATS environment variable, or compiler
the collector without -DSILENT).

			 

			Assuming your application normally uses much
less than 1 GB, possible causes are:

			 

			1) You are allocating large objects containing
seemingly random bit-patterns.  If that's the case, these may appear to
the collector to be pointers.  The C++ proposal addresses this issue
with declaration that allow the implementation to use type information.
For now, the standard solution is to make sure GC_malloc_atomic is used
for those objects.  You should see this as very little memory reclaimed
by the GC and regular heap expansions, with a large final heap size.

			 

			2) You are internally using some form of custom
allocation by obtaining pools of objects from the collector.  This is
generally not a good idea with a garbage collector.  Symptoms will
probably be similar.

			 

			3) This is caused by a single huge allocation
request due to memory corruption.  This should be apparent form the log.

			 

			4) Something else.

			 

			Hans

				 

				________________________________

								From:
gc-bounces at napali.hpl.hp.com [mailto:gc-bounces at napali.hpl.hp.com] On
Behalf Of Simon Tsai
				Sent: Monday, November 13, 2006 4:12 PM
				To: gc at napali.hpl.hp.com
				Subject: [Gc] Out of Memory! Returning
NIL!

				Hi,

				 

				I have integrated gc 6.8 library with my
applications under Windows XP. My system has 1GByte memory. Our
application has used smart heap. The application is muti-thread.  I saw
gc.log has following erros:

				 

				"Out of Memory!  Returning NIL!" and 

				"GC Warning: Thread stack pointer
0x5c9f018 out of range, pushing everything".

				 

				 

				Does anyone know what cause this problem
and how to fix it?

				 

				Thanks,

				 

				 

				simon

				 




More information about the Gc mailing list