Re: Lua and preemptive scheduling for coroutines

[Josh Simmons <simmons.44@...>]

> - Explicit yields are easy to forget at certain places
And easy to add when missed. :) Unless you have no global state you're
going to end up adding some mechanism to your system to stop a
coroutine being interrupted half way through an operation and hence
leaving the global state in a bad way.

> - Code may loop forever, especially a 3rd party untrusted code, which you may want to run by loading scripts
You cannot stop this from Lua (without removing _all_ C functions from
the sandbox), you need to use threading or processes at the OS level
to limit access to resources.

> - If you want to implement something like multi-tenancy, where coroutines belong to different tenants and each tenant has a right for running a certain tenant-specific amount of time (e.g. tenant A can run at most 40% of overal time, and tenants B and C - only 30% each). With ability to interrupt coroutines implicitly it becomes much easier.
> - My previous experience with Java, where only the thread itself may finish itself, shows that it is not always good to rely on the thread/coroutine to obey to the rules, when it comes to giving control to other threads/coroutines. If all of your code follows a strict discipline it works. But once you start using third-party libs or forget yielding at a few places, you get into problems.
>
Repeating myself but unless you don't have any C functions in your
sandbox it's impossible to make guarantees like this using a debug
hook. Any C function can potentially stall the entire thread
indefinitely. Harder to do accidentally but if it's going to bring
down your application then you'll need to address it outside of Lua in
either case.

> As I mentioned, I have something like millions of actors in mind (e.g. one per player or on per session or one per social network user). You cannot map all of them to OS threading primitives anyway. Therefore, you need to re-invent one ;-) And actors do not need real parallelism most of the time. They are really similar to coroutines in this regard.
>
At millions of actors I'm guessing using a debug hook is going to be
really bad for your performance characteristics. I'm not suggesting
coroutines are a bad fit, just that pre-empting them defeats half of
the benefits in order to save a few keystrokes.