I have tried turning on the FIXEDBIT bit on a table value, but it did not give any effect (table is collected at the first GC run after reference to table has been removed from the Lua stack).
So, I've managed to get similar kinds of C-protected GCObject hacks working -- but, I don't use FIXEDBIT directly. To manage tables or closures I hook into lgc.c:atomic(), and execute a custom marking function after the standard markvalue(g, &g->l_registry) call.
That custom mark function is responsible for calling markobject() on all GCObject's that the C code wants kept alive (apart from fixed strings, which can be kept permanently allocated by a call to luaS_fix). It does seem to work. But, I've never tested it while running the GC in generational mode. I'm just running full sweeps at specific times -- incremental collection might introduce complications.
