You don't understand: Lua programs are parsed only once then executed or natively compield from the bytecode only, strings are no longer used except for loading the program the first time, but never when instanciating objects written in that language. There's no marshalling/unmarshaling between Lua functions calls, references are working natively (via indexes in tables stored in the heap or via closures in the stack, it works directly at binary level, and there's never any need to marshall/unmarshall complex data structures or serialize them by building and parsing strings all the time).
I'm not convinced this is the right approach, even when using the LuaC libary, there's a better way to do that without this huge overhead (which also requires lot of heap allocations for these temporary scripts containing everything: the complete source code of all functions the task needs to do, or the code that will use the loader, and all the data they'll work with and that they'll finally return).
We can do that with LuaC by using existing Lua "thread" objects, to which we can attach metadata for adding info about how the threads can operate and can be scheduled.
The LuaC library is needs only to allow creating new threads with theior own Allocator and setup the metadata attached to threads and possibly some "userdata" object to tune the Allocator behavior.
You need however small changes in the Lua bytecode interpret so that it can test a flag indicating that there's a pending interrupt for which the bytecode should stop interpreting continuously, but should insert a call to yield() before continuing with the next instruction. Finally your C library will be used to create the OS-level timer that will just asynchrnously set the interrupt flag, without modifying directyly the state of any running Lua thread. You also need a way for threads to handle some critical sections using some basic test-and-set instruction (this can be implemetned as an unary boolean operator, similar in syntax to "not" except that it must occur before arbitrary expressions but only before a variable name, like with the "local" instruction; this operator can use a new reserved keyword like "testset" or a new lexical token like "?!"; the variable will bet set to a boolean or nil, it does not need a new type, but it will compile a new bytecode instruction which will be atomic and just needs one memory operand or a register index in the Lua closure's window on the stack)
If we don't define this additional bytecode, there's no possibility to synchronize concurrent threads that can preempt the exuecution while working on shared objects, leaving them in inconsistant state (this limits a lot the possility for real multithreading when working with shared data structures, or when performing I/O, so I'm convinced we need some mutex mechanism). Adding thys additional bytecode is a trivial modification to the VM.
However we don't need a pure scheduler with relative priorities between threads. All that is needed is to allow threads to run really concurrently without having them to be modified so they include many "yield()" calls every where in their code or within all their loops or before/after each blocking I/O.
If Lua is really reentrant as stated in its documentation, then even its blocking I/O libary would be interruptible and reentrant, so that we can force them to yield() in Lua instead of blocking inside the OS thread with its own internal yield to give hand to other processes and OS-native threads (but may be this requires modifying a bit the standard I/O library to force them to yield() in Lua instead of blocking in the native C standard I/O library, or in an OS API used by the native C standard I/O libary.
