[Gc] FW: Win32: Multiple threads already spawned prior to GC_init? GC Aborts due to unknown Thread!

Alec Orr Alec.Orr at wbemsolutions.com
Wed Jan 23 06:46:43 PST 2008


Michael:

While we did not face the error message you saw, we were confronted with a 
similar problem:  How can the GC intercept all thread creation calls made by 
foreign libraries so that the GC can properly track/halt threads?  While we 
predominantly use linux, I believe the same paradigm can be applied to Win32.

I realize you are looking for the elegance of a GC_init(), but thought I would 
offer this to you anyway.

1.  We created wrapper functions for pthread_create, pthread_join, and 
pthread_detach.

-  These functions intercept ALL calls to pthread_create, pthread_join, and 
pthread_detach for the entire process (the GC intercepts others as well).  The 
purpose of these functions is to use a conditional to EITHER call the pthread 
version of these functions or the GC versions of these functions.  In other 
words, we are re-routing pthread calls to the Garbage collection equivalents first.

- The difficulty is that the GC_pthread_create WILL eventually call 
pthread_create, so you need to keep track of the original function pointer.  On 
linux, we used a dlsym with RTLD_NEXT to replace the global pthread functions 
with our versions.

2.  For EACH thread that is created, we use pthread_getspecific or 
pthread_setspecific to store a thread local toggle-like variable to track 
whether or not the GC_pthread_create is next in line to be called or the old 
pthread_create should be called.  This is to prevent recursive calls.  Consider 
this:

a.  Some foreign DLL calls pthread_create
b.  my_pthread_create gets called.
c.  my_pthread_create looks for a TLS variable called 'toggle', sees that it 
does not exist, and creates the variable assigning the value to 1.
d.  Because 'toggle' did not exist, my_pthread_create calls GC_pthread_create.
e.  GC_pthread_create will try to call pthread_create to actually create the 
thread.  Since I have replace pthread_create, my_pthread_create wil get called 
again.
f.  my_pthread_create checks the TLS variable and sees that it's set to 1. 
Reset it to zero, and call the old_pthread_create.

I suspect (though I haven't tried) that one could port this paradign to Win32 
equvalent functions (CreateThread, etc).  We use a pthread wrapper library on Win32.

Attached is a (rather dated) sample I submitted back in '04 that illustrates 
this for other people in the same boat (it is linux based unfortunately). 
Rather than using a TLS marker it actually re-implements the functions (probably 
not ideal).

Hope this helps somewhat,
Alec

Michael Paul Brandt wrote:
> Dear Hans,
> 
>  
> 
> We are very curious about your opinion about the matter described in 
> posting below  a few weeks ago.
> 
>  
> 
> This could of course be a Win32 specific issue, but still, it seems very 
> generic by nature. A foreign, multithreaded host program (not using GC) 
> loads a dynamic library based on GC.  The GC knows only of the loading 
> thread and hence the problem occurs when another thread enters GC.
> 
>  
> 
> We have extended the GC with an explicit 
> attachThreadIfNotAlreadyAttached-like function that we call for each 
> thread requiring GC service. This works for us, but in general it is 
> cumbersome and difficult to ensure that this explicit attachment is 
> performed for each thread. We are still hoping for a generic solution 
> where gc_Init ensures that all threads are registered in GC. The only 
> difficulty we can spot is how to acquire the information the GC needs 
> per thread, when not running as them.
> 
>  
> 
> Kind regards,
> 
>  
> 
> *Michael Brandt*
> 
> /Prolog Development Center A/S/
> 
> /Tel. +45 36360000/
> 
> /Mail: michael.brandt at pdc.dk/
> 
>  
> 
>  
> 
>  
> 
> *From:* Michael Paul Brandt
> *Sent:* 3. januar 2008 23:23
> *To:* gc at linux.hpl.hp.com
> *Cc:* VIPDevelop
> *Subject:* Win32: Multiple threads already spawned prior to GC_init? GC 
> Aborts due to unknown Thread!
> 
>  
> 
> Hello all,
> 
>  
> 
> We develop compilers and run-time systems for the Visual Prolog language 
> on Microsoft platforms. We have been using the Boehm-Demers-Weiser GC 
> for the past five years (currently v6.8) as part of our kernel runtime 
> DLL. 
> 
>  
> 
> Currently we are experiencing problems when running a Visual Prolog COM 
> component under Internet Information Server (IIS). Most likely because 
> the GC is called from an unknown thread.
> 
>  
> 
> The details are as follows:
> 
>  
> 
> The GC.lib is compiled with Visual Studio 2005 and flags
> 
>  
> 
> WIN32;NDEBUG;_LIB;GC_WIN32_THREADS;SILENT;NO_DEBUGGING;ALL_INTERIOR_POINTERS;DONT_ADD_BYTE_AT_END;LARGE_CONFIG;VIP_ALLOC_STATISTICS;VIP_RUNTIME;GC_DLL;USE_MUNMAP;VIP_SMART_ROOTS;_CRT_SECURE_NO_DEPRECATE;$(NOINHERIT)
> 
>  
> 
> Threads are attached and detached using the DllMain callback. We have 
> programmed our own VIPGC_register_dynamic_libraries function that 
> minimizes the GC root set such that "foreign" data segments aren't 
> considered roots.
> 
>  
> 
> We develop Visual Prolog COM DLLs. These COM DLLs are accessed from 
> Microsoft Internet Information Server (IIS) using COM Interop from C#. 
> The IIS uses a worker process with numerous worker threads to invoke our 
> COM DLL. Upon Vip7Kernel.dll loading we immediately call GC_init, but 
> only the loading thread gets registered in GC’s thread structure. Now, 
> at some stage another IIS worker thread decides to call some Visual 
> Prolog code using the COM DLL which results in a call to the GC. The GC 
> decides to do some housekeeping and starts the mark phase, which require 
> all thread stacks to be pushed onto the mark stack.  The function 
> GC_push_all_stacks()of win32_threads.c is called and it pushes stacks 
> for all _registered_ threads and examines if current executing thread is 
> among the registered. If so, the variable found_me is set to true. The 
> function completes by verifying that found_me is indeed true or else it 
> aborts!
> 
>  
> 
>   if (!found_me) ABORT("Collecting from unknown thread.");
> 
>  
> 
> This is exactly what we experience in our embedded DLLs, since the 
> requesting thread has never been properly registered in the GC’s thread 
> structure.
> 
>  
> 
> We have been through the GC mail archives and found no mentioning of 
> this thread issue. We have also reviewed the latest v7.0 code of the GC, 
> but this issue seems to remain a problem.
> 
>  
> 
> Would the proper solution (for us at least) be to enumerate all threads 
> of our process (using CreateToolhelp32Snapshot) and attach each 
> explicitly to the GC (right after GC_init is called)? Have anyone 
> encountered similar problems when embedding GC in "foreign" process 
> environments?
> 
>  
> 
> We appreciate all feedback. Thank you.
> 
>  
> 
> Michael Brandt
> 
> Prolog Development Center A/S
> 
> Tel. +45 36360000
> 
> Mail: michael.brandt at pdc.dk <mailto:michael.brandt at pdc.dk>
> 
>  
> 
>  
> 
>  
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Gc mailing list
> Gc at linux.hpl.hp.com
> http://www.hpl.hp.com/hosted/linux/mail-archives/gc/


-- 
Alec Orr
Staff Engineer
http://wbemsolutions.com
978-947-3609
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GCThreadExample.zip
Type: application/zip
Size: 4850 bytes
Desc: not available
Url : http://napali.hpl.hp.com/pipermail/gc/attachments/20080123/f3d69a9d/GCThreadExample-0001.zip


More information about the Gc mailing list