[Gc] collecting based on total heap size

Andy Wingo wingo at pobox.com
Wed Jul 6 02:36:46 PDT 2011


Hi,

Periodically in Guile we see bad GC behavior when a GC-managed object
holds a pointer to a libc-managed object, with a free function in the
finalizer.

I realize that the best thing to do is simply to give libgc control over
the entire heap.  However this is not always possible -- in Windows, it
seems that one cannot override `malloc' and `free'.

It's not even always possible to override the allocators for a
particular library.  For example, in this thread:

  http://thread.gmane.org/gmane.lisp.guile.devel/12552

Mark looks at computing large factorials, using the iterative method,
but it kills the system, as the GMP-allocated memory is unknown to
libgc.  We can't change Guile 2.0's usage of GMP to use the GC malloc,
as other users of GMP in the same process might be expecting to have to
`free()' the data.  Changing it in the development series is also
tricky, as this sort of bug would have to be found somehow.

So, what I wonder is, can we make GC take the total memory size of a
process into account when making the decision to collect or not?  That
would handle the libc malloc case, as well as any other allocator.

The logic could be, "if the process has increased its mapped memory size
by X since the last collection, have GC_should_collect() return TRUE."

X could be an absolute amount or a fraction of the previous heap size.

The condition could be "since the last collection" or "since the last
process-size-triggered collection".

Heap size may be measured directly by libgc, or there could be an option
to use mallinfo, or something.

Dunno.  I wanted to throw these ideas out there before doing any work on
it, as it's a tricky problem and I'm sure yall have thought about it
before.  Thoughts?

Thanks,

Andy
-- 
http://wingolog.org/


More information about the Gc mailing list