[Gc] Thread Local Storage?

The Devils Jester thedevilsjester at gmail.com
Sun Oct 20 12:19:00 PDT 2013


That is exactly how I thought add_roots worked, so I must be missing
something.  Let me explain my understanding of the process, and we can see
where I am going astray.

The GC will collect from areas it "knows" about, (i.e. roots).  In my
situation it is likely that it does not know to collect from the location
that the TLS is placed.  I would use add_roots to tell the GC to consider
this area too.  I would then, before the thread ends, remove_roots to no
longer look at this area?

If my above summary is correct, I come back to my previous question.  If I
tell the GC to look at the TLS area for collection (add_roots) when I
create a new thread, but then I tell it to no longer consider that area
(remove_roots) when I end the thread, but I do this before the data that
was created in that area was collected (so the GC is no longer looking
there, but I still have unfree'ed data there), then would this data not
persist until the completed thread free's its resources?

If my summary is off, which is likely considering the complex nature of
this process, what do I have wrong?


On Sat, Oct 19, 2013 at 5:00 PM, Bruce Hoult <bruce at hoult.org> wrote:

> Roots are areas of memory that the GC searches for pointers at the start
> of GCing, even though it doesn't have pointers to THEM and they may not be
> in the GC heap. You always should have as roots the CPU registers, the
> global variables for the main program and hopefully any libraries, and the
> stacks of every thread. The GC hopefully finds all of those automatically
> (though DLLs and thread stacks can be hard to find on some platforms)
>
>
> On Sun, Oct 20, 2013 at 10:52 AM, The Devils Jester <
> thedevilsjester at gmail.com> wrote:
>
>> I know what the GC does, I think its confusion about what add_roots does.
>> On Oct 19, 2013 4:48 PM, "Bruce Hoult" <bruce at hoult.org> wrote:
>>
>>> There seems to be some confusion about what a GC is.
>>>
>>> The GC will collect and reuse *everything* unless it can see that it is
>>> still wanted because it found a pointer to it.
>>>
>>> The reason to use GC_add_roots() is so that the GC can see your
>>> pointers and won't reuse memory you allocated using the GC. If you don't
>>> call GC_add_roots(), or after you call GC_remove_roots(), the GC is
>>> free to reuse GC memory pointed to from your TLS. That would be bad if you
>>> were still using it.
>>>
>>>
>>> On Sat, Oct 19, 2013 at 7:42 AM, The Devils Jester <
>>> thedevilsjester at gmail.com> wrote:
>>>
>>>> Since a GC can happen at any arbitrary time, what happens if I use
>>>> GC_add_roots on my TLS at the start of the thread, and then GC_remove_roots
>>>> right before the thread exits, but the GC has not collected the data yet?
>>>>  Will the resources continue to pile up until the thread has had its
>>>> resources released?
>>>>
>>>>
>>>>
>>>> On Fri, Oct 18, 2013 at 1:35 PM, Bruce Hoult <bruce at hoult.org> wrote:
>>>>
>>>>> Yes, of course, that's the reason GC_remove_roots() exists.
>>>>>
>>>>> What do you mean "even if GC has not occurred"? That's irrelevant.
>>>>>
>>>>>
>>>>> On Sat, Oct 19, 2013 at 1:11 AM, The Devils Jester <
>>>>> thedevilsjester at gmail.com> wrote:
>>>>>
>>>>>> The number of threads created is not a known value, but I suspect it
>>>>>> will often be very low with longer life spans.  My question is more about
>>>>>> accumulation.  If I create thousands (or hundreds of thousands) of threads
>>>>>> over the lifetime of the program, even if this is days, weeks, or months,
>>>>>> will the performance be impacted by all of the registered areas?  Would it
>>>>>> be safe to 'unregister' an area when the thread is ended, even if GC has
>>>>>> not occurred?
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 18, 2013 at 12:42 AM, Bruce Hoult <bruce at hoult.org>wrote:
>>>>>>
>>>>>>> That all depends on how often you are creating and deleting threads.
>>>>>>>
>>>>>>> If you are creating and deleting thousands of threads between each
>>>>>>> GC then 2) will be lower impact. If threads live for many seconds or
>>>>>>> minutes then 1) will be lower impact.
>>>>>>>
>>>>>>> If you only have a few dozen threads then performance in this area
>>>>>>> is probably irrelevant.
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 18, 2013 at 6:29 PM, The Devils Jester <
>>>>>>> thedevilsjester at gmail.com> wrote:
>>>>>>>
>>>>>>>> The platform that I am currently experiencing the issue on is
>>>>>>>> Ubuntu Linux 64bit 13.04, using GCC 4.8.
>>>>>>>>
>>>>>>>> If I just have a few pointers to some containers, would it be safe
>>>>>>>> to simply add those pointers to the 'roots'?
>>>>>>>>
>>>>>>>> You say that there might be performance problems with approach 1? I
>>>>>>>> only have a few TLS pointers, do you think this would have any kind of
>>>>>>>> performance impact?  Since I (re)instantiate these pointers for every
>>>>>>>> thread, I would assume that I would have to "mark" every instance of these
>>>>>>>> pointers?  Would the performance hit start to grow, then, as new threads
>>>>>>>> are created and points are marked?
>>>>>>>>
>>>>>>>> On a slightly related note, your approach 2 has me wondering if
>>>>>>>> there is some other generic call back (not specific to roots) that I can
>>>>>>>> set for before (and maybe after) a GC occurs.  (It would help debug things
>>>>>>>> that happen specifically at a GC).
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 18, 2013 at 12:09 AM, Bruce Hoult <bruce at hoult.org>wrote:
>>>>>>>>
>>>>>>>>> Fundamentally, someone needs to make sure that the thread local
>>>>>>>>> storage is added to the root areas the GC scans pointers from.
>>>>>>>>>
>>>>>>>>> Ideally, the standard GC would do that. I don't know what your
>>>>>>>>> system is and therefore whether it is practical for the GC to locate the
>>>>>>>>> TLS global areas.
>>>>>>>>>
>>>>>>>>> But you can do it yourself.
>>>>>>>>>
>>>>>>>>> There are two approaches:
>>>>>>>>>
>>>>>>>>> 1) for fixed size areas (or even individual pointer variables) you
>>>>>>>>> can call GC_add_roots(low_address, high_address) on as many areas as
>>>>>>>>> necessary. You can also remove these areas if necessary. Or remove and
>>>>>>>>> re-add if they change size. See gc.h
>>>>>>>>>
>>>>>>>>> 2) for things that change often during execution, it can be better
>>>>>>>>> to point GC_push_other_roots at your own function, which will be called
>>>>>>>>> near the start of every GC. Your function should call e.g.
>>>>>>>>> GC_push_all(bottom, top) on each appropriate area you want the GC to scan
>>>>>>>>> from. Make sure you save the old value of GC_push_other_roots before
>>>>>>>>> installing your function, and call it too. This is all defined in
>>>>>>>>> include/private/gc_priv.h
>>>>>>>>>
>>>>>>>>> Unless you see performance problems, it's safer to stick to
>>>>>>>>> approach 1).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Oct 18, 2013 at 5:20 PM, The Devils Jester <
>>>>>>>>> thedevilsjester at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I have a couple of stl containers that hold pointers to classes
>>>>>>>>>> which inherit from gc.  The containers are created via a template that
>>>>>>>>>> includes the gc_allocator.  This has worked great for quite awhile, however
>>>>>>>>>> I have recently made a couple of the containers thread local using
>>>>>>>>>> __thread.  When I do this, however, (while still in my main thread and
>>>>>>>>>> never having created a new thread), at some arbitrary point (maybe when GC
>>>>>>>>>> kicks in?) my container size goes from whatever it was, to zero.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I can remove the __thread from the containers and everything
>>>>>>>>>> works as desired, for an indefinite period of time, but adding __thread
>>>>>>>>>> back in, always causes the same issue.  Is there some special
>>>>>>>>>> initialization I need to do to allow for __thread (and similar) to work
>>>>>>>>>> correctly in the GC?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> This message has been scanned for viruses and
>>>>>>>>>> dangerous content by *MailScanner* <https://www.mailscanner.info/>,
>>>>>>>>>> and is
>>>>>>>>>> believed to be clean.
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Gc mailing list
>>>>>>>>>> Gc at linux.hpl.hp.com
>>>>>>>>>> https://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> This message has been scanned for viruses and
>>>>>>>> dangerous content by *MailScanner* <https://www.mailscanner.info/>,
>>>>>>>> and is
>>>>>>>> believed to be clean.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Gc mailing list
>>>>>>>> Gc at linux.hpl.hp.com
>>>>>>>> https://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> This message has been scanned for viruses and
>>>>>> dangerous content by *MailScanner* <https://www.mailscanner.info/>,
>>>>>> and is
>>>>>> believed to be clean.
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> This message has been scanned for viruses and
>>>> dangerous content by *MailScanner* <https://www.mailscanner.info/>, and
>>>> is
>>>> believed to be clean.
>>>>
>>>> _______________________________________________
>>>> Gc mailing list
>>>> Gc at linux.hpl.hp.com
>>>> https://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>
>>>
>>>
>> --
>> This message has been scanned for viruses and
>> dangerous content by *MailScanner* <https://www.mailscanner.info/>, and
>> is
>> believed to be clean.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://napali.hpl.hp.com/pipermail/gc/attachments/20131020/03012846/attachment.htm


More information about the Gc mailing list