[Gc] Changing ‘GC_should_collect’ to account for malloc’d memory

[Ludovic Courtès]

Hi,

Andy Wingo’s post at
<http://wingolog.org/archives/2011/02/25/ports-weaks-gc-and-dark-matter>
nicely describes issues that led to the leak of port objects in Guile.
The following excerpt describes a situation, which I think could be
improved:

  Firstly, port objects are more than meets the eye: the eye of the
  garbage collector, that is. Besides the memory that they have that GC
  knows about, they also in general have iconv_t pointers for doing
  encoding conversions.  Those iconv_t values hide kilobytes of
  allocation, on GNU anyway. [...]

  See, the GC only runs when it thinks it's necessary. Its idea of
  "necessary" depends, basically, on how much has been allocated since
  it last ran. The iconv_t doesn't inform this decision, though; to the
  GC, it is dark matter. So it is possible for the program to outrun the
  GC for a while, increasing RSS in the part of the heap the GC doesn't
  scan or account for.

So, how can we make ‘GC_should_collect’ account for memory that’s
malloc’d behind libgc’s back?

A simple and lightweight solution would be to use Glibc’s malloc hooks,
where available, so that malloc calls go through libgc, giving it a
better idea of how much heap is in use (patch attached.)

While it basically works, it seems that uncollectable memory, as used by
the malloc hook, isn’t accounted for by ‘GC_adj_bytes_allocd’.  This
would need to be changed.

What do you think?

Thanks,
Ludo’.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/x-patch
Size: 1623 bytes
Desc: not available
Url : http://napali.hpl.hp.com/pipermail/gc/attachments/20110226/2e9a2bc9/attachment.bin