Re: luaproc thought

[Jerome Vuarand <jerome.vuarand@...>]

2009/9/30 Mark Hamburg <mark@grubmah.com>:
> Having written the multithreading logic for Lightroom and reading through
> the luaproc examples, a thought occurs to me about a potentially useful
> syntactic extension to Lua for these cases. It would be nice to have a way
> to say essentially "the following portion of the program text should form a
> syntactically valid Lua chunk which should be encoded as a loadable string".
> For example, let's look at the example from the luaproc article
> (http://www.inf.puc-rio.br/~roberto/docs/ry08-05.pdf):
>
>        luaproc.newproc( [==[
>                -- create a new channel
>                luaproc.newchannel( "achannel" )
>                -- create a new Lua process
>                luaproc.newproc( [=[
>                        -- send a message to the channel
>                        luaproc.send( "achannel", "hello world" )
>                ]=] )
>                -- create a new Lua process
>                luaproc.newproc( [=[
>                        -- receive a message from the channel
>                        msg = luaproc.receive( "achannel" )
>                        -- print the received message
>                        print( msg )
>                ]=] )
>        ]==] )
>
> This requires careful use of the string nesting markers and none of the code
> can be syntax checked until execution. Furthermore, when we do execute it,
> we have to compile code for every new Lua process. Presumably we could get
> around the last point by caching the results of a loadstring somewhere.
>
> We could get around the lack of a syntax check by using special marker
> characters for the string and having an IDE pull them out and check them
> separately, but that requires an IDE doing a separate syntax check. We might
> really prefer to check it all in the initial load. In particular, consider
> the fact that if the inner calls to newproc are handed invalid code the
> error happens in the outer process with no good way to communicate with the
> root environment.
>
> We could get around this by passing functions and dumping them to bytecode
> strings. That, in fact, is what Lightroom does. However, we have to be
> careful to ensure that the functions so used do not have any upvalues even
> as they are being created in the midst of code that values upvalues for
> performance. One could copy the upvalues over but that requires a more
> sophisticated copying mechanism and if it can't copy everything we're back
> to potentially not checking inner process creations until it's awkward to
> report failure.
>
> But imagine a chunk keyword (anathema I know to add a keyword, but imagine
> it none the less). This keyword basically works just like "function" but it
> returns the bytecode for the chunk and does not capture upvalues. Now we
> write:
>
>        luaproc.newproc( chunk()
>                -- create a new channel
>                luaproc.newchannel( "achannel" )
>                -- create a new Lua process
>                luaproc.newproc( chunk()
>                        -- send a message to the channel
>                        luaproc.send( "achannel", "hello world" )
>                end )
>                -- create a new Lua process
>                luaproc.newproc( chunk()
>                        -- receive a message from the channel
>                        msg = luaproc.receive( "achannel" )
>                        -- print the received message
>                        print( msg )
>                end )
>        end )
>
> The differences are subtle, but the problems cited above are resolved.

Don't you think that it would be very confusing that some code bits
have upvalues, but others do not, all mixed up in the same file ? That
is amplified by the fact that the 'chunk' code of the example could
try to access outer variables, but instead of upvalues you would get
perfectly legal global variable access, and thus no way to report that
as an error at compile time.

I'm not agains the proposal, but I think the limit should be something
more visible than a keyword. A new type of pseudo-string quote pair
would be more appropriate imho. For example it could be a derivative
of [=[ ]=], with the = character replace with something else. For
example [=[return 42]=] is a simple string, but [x[return 42]x] is
string whose content must be a valid Lua script. This would still
allow nesting just like long strings by duplicating the middle
character.

Besides that, there may be a technical limitation that would make that
addition (with your syntax or mine) difficult. For the moment, a Lua
script is always parsed entirely in one step (a single call to
lua_load). Allowing "embedded but independent" scripts would imply
that the parser of the current script could instanciate another clean
parser to parse the substring, eventually forward the syntax errors,
or otherwise continue with the outer script parsing. I don't know the
Lua source code, but depending on the precise Lua implementation, that
might be complicated to do.