Re: Guide for Lua bindings

I like the fifth approach. I'm a bit biased because I'm the one who wrote sol2, but higher-level abstractions are almost always infinitely more useful.

But, the main objective of this thread (I believe) is moreso something that teaches you about the ins and outs of using the Lua C API, and any quirks that come with it. In my opinion, the manual is your best friend here. When you're programming against any API, you need hard-guarantees about what individual function invocations do. The state it leaves things in, how it interprets arguments (if it's non-obvious), and other such things. Any higher-level tutorial likely won't serve you as well as just knowing the manual: plus, it's much easier to hold a developer accountable to their documentation, (well-written, short) examples and manual than to a general best-practices guide.

The only thing I found missing in the manual is some of the behavior of the default Lua's Coroutines. For example, when you do

runner = coroutine.wrap( ... ) -- or
runner = coroutine.create( ... )

The C library creates a new lua value of type LUA_TTHREAD, pushes the function you pass to invoke into that Lua Thread's stack, and then leaves it as is. The first time you invoke runner(), it will pull that function off the stack. Then, subsequent invocations will use {implementation defined magic} to keep track of that function and run it on the coroutine's thread even though it's disappeared of the Lua Thread's execution stack after the first time you run it (that is, it's no longer accessible to you in an obvious way).

What's fun is wrapping your head around that and then also learning that lua_resume (https://www.lua.org/manual/5.3/manual.html#lua_resume) does not work like lua_call or lua_pcall despite having an incredibly similar signature and usage notes: the latter 2 functions pull and call your argument from the top of the stack and generally don't mess with things below that (sans direct user invervention).

lua_resume reserves the right to own the thread's entire stack, and thusly will actually clear things off the stack below the nargs + function you push onto the stack to make lua_resume work (in fact, it's questionable whether you need the function there either, but at present sol2 pushes it onto the stack anyhow). This means that any abstraction you make around `coroutine` isn't as simple as just duplicating whatever `function` abstraction you have around `lua_pcall`: you instead have to take into account that the entire stack can (and generally, is) owned by the entire execution thread, and it's liable to clear the entire thing by the time `yield()` is called.

Just some quirks I picked up, but honestly they're not really all that important in the grand scheme of things. Coroutines and how they work were the most difficult to understand, not because I mistaked them for operating system threads, but because I didn't quite grok that in this case "Coroutine" and "Thread" just mean "Separate Execution Space". Using that terminology, it makes a lot more sense.

On Fri, Dec 22, 2017 at 12:21 PM, Kalafut, Bennett <bennett.kalafut@thermofisher.com> wrote:

A fifth approach is to use third-party bindings for standard Lua (no stuck-in-5.1 LuaJIT needed) like Sol2.

From: Dirk Laurie [mailto:dirk.laurie@gmail.com]
Sent: Thursday, December 21, 2017 8:34 PM
To: Lua mailing list
Subject: Re: Guide for Lua bindings

2017-12-22 2:54 GMT+02:00 Gregg Reynolds <dev@mobileink.com>:
> A humble suggestion for Roberto (or anybody else): a thorough guide to Lua
> as extension and extending Lua.
>
> I just finally paid good $$ for PIL 4, and am glad I did. However, it's main
> focus, naturally, is Lua programming. The material on the C interface is
> good as far as it goes, but it leaves me hungry for more.
>
> I would pay good Bitcoin for a Guide to writing Lua bindings that covers the
> hairy bits - like dealing with coroutines, strategies for organizing code,
> etc. - in a little more detail, with examples, maybe covering multiple
> languages.
>
> There's plenty of info along these lines on the web, but it sure would be
> nice to have an authoritative Guide.

There are at least four approaches:

1. Use a tool that generates bindings for you directly from the headerfiles.

2. Use LuaJIT which has an FFI built in.

3. Do it neatly by hand but stay so close to the the original that the C documentation can be used as is. Most of the better contributions to LuaRocks do this.

4. Hide all the details under a level of abstraction so that the user need not even know which library is being bound. This approach sits underneath Love2D.

Each of these has advantages that in certain situations makes it more approprate tham the others.

I think it is a good thing, and typical of the Zen of Lua [1], that the Lua team does not enshrine any of the above.

[1] Less is more.