pinning objects in ffi.gc() finalizers

[Cosmin Apreutesei]

Hi,

I noticed that pinning objects in ffi.gc finalizers() doesn't work, at
least not when the program exits -- it's like the upvalues of the
finalizer are weak references. Is this a bug or a feature?

Example:

local heap = ffi.gc(CreateHeap(), FreeHeap)

local mem = ffi.gc(CreateMem(heap, size), function(mem)
FreeMem(heap, mem) -- heap pinned in mem's finalizer
end)

When the program exits, sometimes heap's finalizer is called before
mem's finalizer, even though mem's finalizer pins heap.

[Mike Pall]

Post by Cosmin Apreutesei
I noticed that pinning objects in ffi.gc finalizers() doesn't work, at
least not when the program exits -- it's like the upvalues of the
finalizer are weak references. Is this a bug or a feature?

'pinning' (in GC terminology) means preventing an object from
moving its address. That's not applicable here, since Lua and
LuaJIT have a non-moving GC.

You probably mean anchoring. But upvalues are strong anchors, so
this is not what happens here.

Post by Cosmin Apreutesei
When the program exits, sometimes heap's finalizer is called before
mem's finalizer, even though mem's finalizer pins heap.

Don't confuse finalization and freeing of the underlying objects.
In this case it's the cdata pointers themselves which are most
certainly anchored and kept alive by the upvalues.

But that doesn't affect finalization order: cdata objects do not
have a defined finalization order right now (unlike userdata).
Actually, cdata finalizer handling is rather messy in the current
GC and fixing that would be tricky.

[
Note that you can create a similar problem with userdata where a
finalizer of an older object has an upvalue anchoring a younger
object: that doesn't prevent the younger object from being
finalized first -- but it does prevent the younger object from
being freed before the finalizer for the older object is invoked.

local o1 = newproxy(true)
local o2 = newproxy(true)
getmetatable(o2).__gc = function(x) print("finalize o2", x) end
getmetatable(o1).__gc = function(x) print("finalize o1", x, o2) end
-- This finalizes o2 first, even though o1's finalizer anchors o2.
]

--Mike

[Cosmin Apreutesei]

Post by Mike Pall
You probably mean anchoring.

Yes, sorry I didn't know "pinning" was a technical term, I used it
colloquially to mean "keeping a (strong) reference to" in a Lua
context.

Post by Mike Pall
But that doesn't affect finalization order: cdata objects do not
have a defined finalization order right now (unlike userdata).
Actually, cdata finalizer handling is rather messy in the current
GC and fixing that would be tricky.

Thanks, that's good to know. IMHO it would be useful to document this
(intuitively I expect to not be able to access finalized objects,
ever).

Post by Mike Pall
Note that you can create a similar problem with userdata where a
finalizer of an older object has an upvalue anchoring a younger
object: that doesn't prevent the younger object from being
finalized first -- but it does prevent the younger object from
being freed before the finalizer for the older object is invoked.
local o1 = newproxy(true)
local o2 = newproxy(true)
getmetatable(o2).__gc = function(x) print("finalize o2", x) end
getmetatable(o1).__gc = function(x) print("finalize o1", x, o2) end
-- This finalizes o2 first, even though o1's finalizer anchors o2.
]

Right, that's an effect of the two-pass finalize-all then free-all Lua
semantics, which is a tad better than undefined-order but still
doesn't solve it 100%, which is what you clearly show with this
example. I agree, we need a 100% solution to this, Lua finalizer
semantics is a cop out :)