Re: How Lua code can be updated at runtime without breaking global state?

[Graham Wakefield <wakefield@...>]

Here's a few things we've learned using LuaAV for live coding. We tried this both with reloading a modified script into the same lua_State (reload-style), and also with running new chunks into the same lua_State (console-style). The most important insight is to break the local habit.

Usually you get into the habit of making variables local, for speed etc.; but this means they will no longer be present when the next chunk is run, and you can easily end up with functions that refer to different upvalues of the same name, which can be very confusing. To persist data between entries, we got into the habit of <global> = <global> or <init>, e.g. "g = g or 10" (as Vadim suggested). Easy to reset this with "g = nil". In console-style it is also important to avoid the habit of local references to functions, modules etc. "local socket = require 'socket'" will only work in the chunk it runs in. Console-style code updates hardly ever use locals. We also had to be careful with dangling coroutines, so "co = co or coroutine.wrap(...)" is another one to keep in mind. This gets more complex when coroutines are attached to some kind of scheduler. The chances are you want to change the implementation of a function without losing the progress it has made, which can't be done without introducing some indirection. In that case using an explicit call to a global function name is the only way, e.g.:

-- a global that can be re-implemented live due to explicit recursion:
function foo() 
	... do stuff
	coroutine.yield()
	return foo()
end

Obviously using a table index is another option (that's what globals are anyway...)

In summary, unfortunately a lot of the advantages of locals such as speed, closures to lexical scoping, don't translate well to live coding and dynamic updates (especially console-style), where some kind of global indirection is necessary for the dynamic intercession. Our most recent live coding experiments have shifted to using more declarative problem-oriented syntaxes built on top of Lua, such as expression object constructions and jQuery-style select/map applications.

On Mar 26, 2013, at 2:21 PM, Paul K wrote:

> Hi Nikolay,
> 
>> I am wondering, what techniques are used to dynamically update Lua code without breaking normal execution of a program?
> 
> You can't really update Lua code without breaking "normal" execution.
> The code that is responsible for reloading code fragments needs to be
> executed somehow. In some environment there is an event that triggers
> a reload (for example, it checks timestamps of Lua scripts) and in
> other cases the application is controlled from a debugger, which
> reloads the script when the source changes.
> 
> I've described several high-level mechanisms I use in ZeroBrane Studio
> to support live coding here:
> http://stackoverflow.com/a/10937422/1442917. The way ZBS does it is
> simple: it detects a change in the source and sends a command to the
> debugger to stop the application; it then sends the update fragment to
> be (re-)evaluated in the context of the running application; and then
> continues the execution of the application.
> 
>> There are a lot of caveats, for example what if program stores some non-constant data in global variables that are initialized in the beginning of the script? Simply reloading the script will reset that global variables to initial values...
> 
> As Vadim suggested, it's possible to do "value = value or default" if
> you do need to re-execute this fragment. If's also possible to put the
> initialization code inside "if not initialized" check where
> "initialized" is also some global variable.
> 
>> Otherwise, the only real problem are upvalues, as those can't be updated (or
>> easily?).
> 
> Upvalues can present problems, but it's possible to recreate them.
> Shared upvalues present bit more challenge: I don't think there is a
> way to join them in Lua 5.1, but it's possible in Lua 5.2. In Lua 5.1
> one may need to recreate all functions that may have joined upvalues
> so that they are indeed shared after re-evaluation (although I haven't
> tested if they really stay shared in this case).
> 
> ZBS doesn't deal with upvalues, but does support live coding for
> various Lua engines; it supports two modes: in one case the entire
> application is restarted and in another only selected fragments are
> re-evaluated, but the application continues. The latter is useful when
> the Lua script is controlled by some other engine (as happens in
> Love2d, Gideros, Corona, Moai, and other game environments), which
> allows code change that still preserve the overall application state.
> For those interested in more details, I have several demos that show
> live coding with Lua engines:
> http://studio.zerobrane.com/documentation.html#live_coding
> 
> Paul.
>