Re: Lua Needs

[Philippe Verdy <verdy_p@...>]

Note, my use of "TLS" here is a bit abusive, because it is not strictly thread-local: objects referenced in closures or function environemnt is accessible in other threads by communicating them via yield(...) or resume(...), because they are all allocated on the same heap shared by all threads/coroutines created from the first one which is started by the instance of the Lua engine (which runs all its "thread" objects in the same OS thread).

But "TLS" or "thread local storage" must be understood by interpreting the "thread" term as being "Lua thread/coroutine", where each one can have its own separate storage; there's no limitation however for allowing other threads to use and modify these storages if they are exchanged by yield(...) and resume(...), passing their references via the stack of each thread. Stacks are very fast compared to the heap and you can even get much higher performance by using yield() and resume() to invoke threads running a service loop, instead of calling functions and passing references by closures.

Le ven. 30 nov. 2018 à 15:39, Philippe Verdy <verdy_p@wanadoo.fr> a écrit :

For this reason, the generalization of the ":" pseudo-operator of Lua is not easy: it would create lot of overhead at compile-time and runtime to create many closure objects, that are compiled using a static "prototype" which is an  integer-indexed array mapping the numbered upvalues to their bound object, and then used at runtime to bind each actual upvalues to other accessible external objects

The ":" pseudo-operator is a syntaxic sugar of Lua which can always be rewritten using the "." pseudo-operator.

But as well the "." pseudo-operator is a syntax sugar of Lua which can always be rewritten using the "[]" table indexing operator (this one is the only real operator).

Combining the two sugars (pseudo-operators ":" and ".") needs to be limited to avoid ambiguities.

The only safe way (without needing the overhead of instanciating too many additional closures) is to declare local variables in separate statements to precompute the object references you need to pass further down for forming the function call. You can finally always resolve all cases using the "[]" table indexing operator only.

Then use the final "()" function call operator where you'll pass the explicit references by these temporary variables and minimize the number of needed closures (because they are costly on the heap, and may then be quite slow if closures are used extensively in tight loops with many repetitions).

---

Note that each closure is an actual object (a simple indexed array, not a full table with random keys) that is allocated on the heap (then instanciated by binding the references to the actual variables), so it has a runtime cost on the garbage collector, each time we call any Lua function that uses any upvalues or returns a list with more than one value).

This object needs then to be garbage-collected when exiting the closure (after each function call). No such closure needs to be allocated is the function has no use of external variables in its lexical scope and uses only its parameters and the reference to its own metatable which keeps data across all calls from the same thread.

The function's own metatable is then like a "thread local storage" (TLS) that does not need to be allocated at each call; this TLS is used only for data that can be modified, it does not contain static data which is stored in its constant pool, including the static prototype built by the Lua compiler and which is used to instanciate the mapping for the closure of the function to its lexical context). This metatable also contains a reference to its external dynamic environment (not bound directly by the closure). C functions are a bit different because instead of a full table, their environment is an indexed array of userdata, but userdata bound to c functions are also thread local storage, allocated on the heap only once per thread. These TLS avoid the extra cost of closures at each call by limiting a lot the use of heap allocation.

If you have a program that makes many loops across function calls and you see that it uses lot of heap and garbage collector is too much sollicited, it may help to see if you can avoid the closures and use TLS instead, i.e. the environment of functions (which is unique per thread, and kept across calls withoout being allocated and garbage collected at each call like closures).

Le ven. 30 nov. 2018 à 14:48, Philippe Verdy <verdy_p@wanadoo.fr> a écrit :

Yes, that's exactly what I understood: you have a variable/conditional "path" to the property of object "a" that interest you, and then "d()" is a method on that property that must be used where you want to it to act on the object "a".

My approach in pure Lua, using a closure to enclose the value of "a", plus a function in Lua to build an intermediate object should work for your case even if you modified it:

    a:[test and 'b' or 'c'].d("additional parameters:", {12, 34}, 'sample') 

which is also writable in Lua as:

    (function(a)

        local __o__ = a;

        return function(...)

            return  (__o__[test and 'b' or 'c']).d(__o__, ...)

        end

    end)(d,"additional parameters:", {12, 34}, 'sample')

 

See how I added also the other parameters (which are passed to the internal function here using a vararg, but the vararg is not mandatory).

The typical usage would be to create dynamic properties in a OOP language with facets:

    value = (some _expression_ selecting an object):[some _expression_ computing a property name].get();

    (some _expression_ selecting an object):[some _expression_ selecting a property name in the object].set(value);

which can be written as:

   value = (function(o)

        local __o___ =  o;

        return function() return  __o__.get(__o__[some _expression_ selecting a property name in the object]) end

    end)(get, some _expression_ selecting an object);

   (function(o, v)

        local __o___ =  o;

        return function() return  __o__.set(__o__[some _expression_ selecting a property name in the object]) end

    end)(some _expression_ selecting an object, value);

There's many variants possible depending on which part is variable or not or dependant on the object that the "apparently simple" syntax

    a:b.c(d)

does not disambiguate clearly. And this would be even worse if we allowed this:

    a:b:c.d(e)

which could be understood differently as any one of:

    ((a:b):c).d(e)

    (a:(b:c)).d(e)

    a:((b:c).d)(e)

depending on associativity, and which object must be passed to the final dynamic method named "d" here.

Le ven. 30 nov. 2018 à 02:56, Coda Highland <chighland@gmail.com> a écrit :

On Thu, Nov 29, 2018 at 11:08 AM Philippe Verdy <verdy_p@wanadoo.fr> wrote:
>  local a = a
>  local x = test and a:b or a:c
>  x:d()

I mean...

a:[test and 'b' or 'c'].d()

/s/ Adam