[Gc] "Too many heap Sections" using allocations that span multiple pages.

Bruce Hoult bruce at hoult.org
Fri Jan 4 15:14:40 PST 2008


On Jan 5, 2008 8:40 AM, Alec Orr <Alec.Orr at wbemsolutions.com> wrote:
> Good Day everyone:
>
> We are doing stress testing of a multithreaded application using the GC 6.8 on
> RH/SLES-10 Linux and Win32.  We are seeing a "Too Many Heap Sections" error on
> Win32 during stress testing, we also see this error in
> real world testing but it takes several days (or week) to hit.

The number of heap sections is just a compile-time constant used to
size a global array.  It can be trivially increased or with just a
change to a few lines of code changed to a dynamically-allocated
resizable array.  Just change to GC_heap_sects (ok,
GC_arrays._heap_sects) to be a pointer and  GC_add_to_heap() to do a
realloc() or similar on it instead of blowing up.


> During stress testing, our application has a need to intermittently allocate
> very large blocks of memory (anywheres between 1 page, or as large as 200 - 500
> MB) as we build up an XML response.

This will cause severe fragmentation for any non-compacting memory
allocator (including malloc()).  I'd really recommend that you limit
your block size to some reasonable value and use a list or array or
tree of blocks to store such huge strings.


> We can alleviate the issue completely by coding the large blocks to come from
> another heap (the C-Runtime rather than the GC).  It appears that when a very
> large block (greater than 1 page) is returned to the GC, that the GC may use the
> large block as a new root for future, smaller allocations (please correct me if
> I am wrong here).

The GC has a setting that controls whether blocks of memory greater
than the GC page size are reserved for future possible allocations of
the same size, or broken up to fulfill smaller allocations when memory
is otherwise exhausted (and the GC would otherwise ask the OS for
more).

See: GC_use_entire_heap


More information about the Gc mailing list