Re: Yielding across lua_load

[Sean Conner <sean@...>]

It was thus said that the Great Soni L. once stated:
> 
> 
> On 11/10/15 04:09 PM, Tim Hill wrote:
> >>On Oct 10, 2015, at 8:07 PM, Soni L. <fakedme@gmail.com> wrote:
> >>
> >>
> >>
> >>On 10/10/15 11:51 PM, Tim Hill wrote:
> >>>>On Oct 10, 2015, at 6:57 AM, Christian N. <cn00@gmx.at 
> >>>><mailto:cn00@gmx.at>> wrote:
> >>>>
> >>>>On 2015-10-08 19:48 +0200, Soni L. wrote:
> >>>>>
> >>>>I don't think that would be possible, as yielding would jump outside 
> >>>>lua_load using longjmp, losing any information lua_load has stored on 
> >>>>the C stack (probably even leaking memory). However, using something 
> >>>>like the C++ library Boost.Context or Boost.Coroutine, a workaround 
> >>>>might maybe be possible as these libraries provide a way to sort of 
> >>>>save the C stack.
> >>>More to the point is why the original OP feels yielding during 
> >>>lua_load() is necessary.
> >>>
> >>>—Tim
> >>>
> >>You can't yield during string.gsub callback either, but that's not 
> >>relevant.
> >>
> >>I actually do have an use for yielding during lua_load() and it has to do 
> >>with a Minecraft mod called OpenComputers. I can't remember what it was 
> >>because it happened a while ago but it did stop me from finishing what I 
> >>was doing.
> >>
> >>It's also useful for JIT-tracing to Lua. (easier to reset/use as you 
> >>don't have to worry about state as much) Especially so when combined with 
> >>a 3rd party cooperative multitasking library. (you wouldn't be able to 
> >>keep the other threads going unless you yielded, which would kill the 
> >>load() and possibly the whole thread as well, because it would error)
> >>
> >If you are worried about compiler overhead, why not spin up a dedicated 
> >Lua state on its own thread and use that as an async compile engine? You 
> >can shuttle the result back as bytecode and then feed this to lua_load() 
> >for very fast load times.
> >
> >—Tim
> >
> >
> >
> When did I say anything about compiler overhead?

  I think what Soni wants (and please, correct me if I'm wrong) is something
along these (hand-wavy) lines:

	function loadcode()
	  local function getdata()
	    local line = coroutine.yield()
	    return line
	  end

	  return load(getdata)
	end

	co = coroutine.create(loadcode)

	for line in io.lines(somesourcefile) do
	  coroutine.resume(co,line)

	  if coroutine.status(co) == 'dead' then
	    error("dead coroutine")
	  end

	  -- do some other processing, maybe run other
	  -- other coroutines as well.
	end

	okay,code = coroutine.resume(co,nil)
	f = load(code)
	f()


  Although there is a workaround:

	function loadcode(code)
	  local data = coroutine.yield()
	  if not data then
	    return code
	  else
	    code = code .. data .. "\n"
	    return loadcode(code)
	  end
	end

	co = coroutine.create(loadcode)

	for line in io.lines("b.lua") do
	  coroutine.resume(co,line)
	
	  if coroutine.status(co) == 'dead' then
	    error("dead coroutine")
	  end
	
	  -- do some other processing, maybe run other
	  -- other coroutines as well.
	end
	
	okay,code = coroutine.resume(co,nil)
	f = loadstring(code)
	f()

  Not quite as elegant, but it works.

  -spc