[Gc] why USE_MUNMAP isn't the default for the collector? (repost)

glauco at virgilio glauco.masotti at virgilio.it
Thu May 4 00:41:30 PDT 2006


Hello. I am reposting a text-only version of my previous message, as nothing of the former appeared.

****************************************************************************

Dear Hans,
you may remember me as I am cited, in the README.changes file, for a small contribution that I gave in 2003.

Although I have been following with attention the developments of your garbage collector software for many years,
I had few occasions to effectively use it, mainly because I had to work in the framework of very large applications
already developed with a malloc/free approach.

In the latest 3-4 years I could move more freely on applications developed newly from scratch.
On one of these I opted for manual memory management after experimenting with your collector for a while, because I 
found manual management more efficient.
This application manages a lot of temporary small structures, but also maintains some large arrays of structures for 
reuse in subsequent runs.
The GC version was a 10-20% slower. However I had recently reconsidered this and after some reorganization of the code 
the result turned in favor of the GC.
In particular it has been decisive to split the structures stored in the large arrays, putting apart their pointer-free 
elements,
in order to make use of GC_MALLOC_ATOMIC for the large arrays of them.
The version with the GC is actually from 10 to 30% faster, and is more successful in reclaiming memory and in keeping 
heap size small,
so I decided, from now on, unless new facts changes this situation, to rely steadily on the GC.
However all the above is true if I use the GC with the USE_MUNMAP option enabled, otherwise the heap used is much larger 
and it tends to grow almost monotonically.
I am thus wondering why USE_MUNMAP isn't the default for the collector? Moreover in version 6.6, it won't even compile 
unless you also enable USE_MMAP (!?).
This at least is what happens under Windows (I am using the GC on Windows in a VC++ environment, linking the GC as a 
static library).
In the previous version that I used (6.2) wasn't like this, so the first time I saw this I wonder if the USE_MUNMAP 
option were still working or not.
You will find attached some snapshot of memory allocation that I made running my application in the same situation,
using version 6.2 and 6.6 with or without the USE_MUNMAP option enabled. These images shows clearly what happens,
it also seems that version 6.6 manages the heap a little better, it allocates less memory and heap size appears to grow 
and shrink more regularly.
Is there any drawback in enabling this option? Is there something wrong in my application that I need to use this 
option? Or is it a problem that arises in Windows only?

Having stated briefly the above as the main question of the present query, there are a number of aspects that I think 
are worth to be examined in more detail.
I hope you will find them interesting and I will appreciate your related comment or advice.

First of all I have to explain how all this started.
The previous version of my application, with malloc/free, was checked for leaks using version 6.2 of the GC, and there 
were none, but after some months and many changes
I noticed an increase in allocated memory after subsequent runs of the application. Obviously I suspected  that some 
change gave rise to a leak,
but despite my efforts and many investigations, with or without the help of the GC, I was unable to find such a leak.
The collector didn't find any leak, it was only giving those obscure messages of the type: "Leaked composite object at 
start: 0x..., appr. length: ...".
By reviewing the posts I found that other users have reported this, but you didn't clarify why the collector reports 
these messages which are apparently meaningless.
Also checking with repeated calls of GC_generate_random_backtrace, everything that was found in memory had reasons to be 
there, thus I found no useless retained reference.
At this point I didn't know how to get rid of this behavior that at the beginning of each run was steeling about 10 MB, 
increasing each time the heap size.
Luckily I found that linking with the GC the heap size remained stable! Thus either a misterious leak existed and the GC 
is better in reclaiming memory than
the manual malloc/free management, or it is able to use memory more efficiently than the standard allocator... or else 
what?.
Therefore I decided to use the collector if I could be able to improve speed at least at the same level of the manual 
memory management.
As it happens sometimes, not all what seems bad comes to harm, and at the end I now have a faster application which uses 
less memory!

However, as it was not simple to get here, and I made many trials, in addition to what I said above, may be that someone 
is interested in what else I found out,
and maybe that you want to confirm if what follows makes sense and corresponds to your experience.

1) USE_MUNMAP introduces a penalty of about 1-2% in total execution time, but, as I said above, it is essential to keep 
heap size small.
2) Contrarily to what I expected, using LARGE_CONFIG is faster.
3) Using the IGNORE_OFF_PAGE versions of malloc and malloc_atomic for large arrays keep the collector silent (otherwise 
it warns with:
"Repeated allocation of very large block (appr. size ...): may lead to memory leak and poor performance"),
but it is a little bit slower than using normal GC_MALLOC or GC_MALLOC_ATOMIC and explicitly deallocate them with 
GC_FREE,
this approach could perhaps be preferred if these large arrays are only a few, isn't it?
4) I build my application linking the GC in VC++ as a static library built using the multithreaded standard libraries, 
as I found this solution
slightly faster than building with single threaded versions (even if I don't use threads).
5) Version 6.6 is slightly faster (1% or less in final timing) than 6.2 (not bad :-).

Thank you for your attention so far.
Best regards,
--- Glauco Masotti



More information about the Gc mailing list