[Gc] Allocating Executable Memory

BGB cr88192 at hotmail.com
Mon Jul 26 08:31:10 PDT 2010

----- Original Message ----- 
From: "Andrew Haley" <aph at redhat.com>
To: <gc at linux.hpl.hp.com>
Sent: Monday, July 26, 2010 7:55 AM
Subject: Re: [Gc] Allocating Executable Memory

> On 07/26/2010 03:37 PM, BGB wrote:
>> ----- Original Message ----- From: "Andrew Haley" <aph at redhat.com>
>> To: <gc at linux.hpl.hp.com>
>> Sent: Monday, July 26, 2010 6:29 AM
>> Subject: Re: [Gc] Allocating Executable Memory
>>> On 07/24/2010 11:04 AM, Ivan Maidanski wrote:
>>>> Sat, 24 Jul 2010 09:50:38 +0100 Andrew Haley <aph at redhat.com>:
>>>>> I think you'll find this problematic in practice, at least on
>>>>> GNU/Linux.  Some distributions forbid the allocation of memory that is
>>>>> mapped writable and executable, and I think the number of such
>>>>> distributions will increase over time.  On such systems Java is marked
>>>>> with a special file attribute, and that attribute is writeable only by
>>>>> root.
>>>>> The way around this is supposed to be to map pages twice, with
>>>>> writable and executable permissions.
>>>> Okey. Another patch is needed which adjust the method how mmaping is
>>>> done.
>>>> If I understand correctly, we just need to duplicate all relevant
>>>> mmap and munmap calls (RW + E), right?
>>>> If yes, we also need some config option to enable this feature
>>>> (otherwise retain old behavior).
>>> Unfortunately, it's harder than that.  On such systems, the only way
>>> to solve the problem is to map each page twice, at different
>>> addresses.  As far as I know, the only way to do that is to use mmap()
>>> on a named file on some device.
>>> In practice the easiest way to do this is to map the pages
>>> independently of the gc and use finalizers to recycle pages once
>>> they're no longer reachable.
>> how about mode changing?...
>> like, a call is used to mark the memory object writable, and another to
>> mark it executable? (via mprotect or similar...).
> That works too.  It's kinda hard if some other thread is executing
> that page, though.

yeah, this is a possible cost.

this would either require complicated "memory scheduling" (or, putting each 
object on its own page, which would be wasteful), or pausing any other 
threads during this operation (admittedly, I haven't looked into how one 
would go about enumerating and pausing/unpausing threads on Linux).

another possibility could be to have a signal or exception handler, which 
would then attempt to stall faulting threads until the operation completes 
(admittedly, this one would probably be cleaner via SEH than via signals, 
since with a signal someone else might want to handle pagefaults, and signal 
handlers aren't really nestable AFAIK...). (with signals, someone might 
override a handler and fail to call the old one if they don't recognize the 

with SEH at least, one possibility would be to check the address, and 
determine if it is in a locked region, and if it is, to wait until this 
memory returns to normal operation (via sleeping or similar), and when this 
happens, to return to normal operation. this would likely be a similar 
mechanism to implementing a write barrier (or certain forms of shared 
memory). otherwise, they pass the exception along.

possibly, even, the mode-changing could be driven by such a handler, if 
needed (although, this could kill performance whenever execution and writing 
are taking place in such memory at the same time).

>> admittedly, I don't really know why distros would make this restriction,
>> as personally it seems restrictive and ill-concieved (and liable to
>> break many otherwise functional apps).
> Surprisingly, it breaks very few apps, because very few apps ever need
> to do this sort of thing.  However, it prevents the use of one of the
> most common techniques used by security exploits.

this should still be a matter of app policy, rather than OS policy.

if the app doesn't need RWX memory, it should not allocate it, and if it 
does, then this is the apps' problem.
and, then, if the app does actually request RWX access, then the OS should 
give it such.

this is a different matter than the OS having the X attribute actually do 

for example, the stack and heap can be marked RW by default, and RWX memory 
primarily given only on explicit request (this being the general policy I 
had taken with my own GC at least).

granted, it is always possible some malloc implementation or similar is 
allocating RWX by default.

>> one could almost just ignore the issue and make the app break on
>> these targets, as a sort of protest against this "feature" (users
>> have to patch themselves, and one can refuse to accept patches to
>> "fix" this feature as a matter of project policy), and if the apps
>> become popular, then it puts pressure on them to forsake such a
>> feature.
> Yes, it is a pain, and I accept your point in general.


personally, I am unlikely to address this matter, but OTOH, if people 
actually used my stuff enough to care, this would itself be surprising.

More information about the Gc mailing list