Re: Lua 5.3 Suggestion: extend lua_(get|set)uservalue() to associate userdata-to-userdata

[William Ahern <william@...>]

On Mon, Dec 02, 2013 at 05:16:58PM -0500, Sean Conner wrote:
> It was thus said that the Great Philipp Janda once stated:
> > Am 02.12.2013 07:01 schr?bte Sean Conner:
> > >It was thus said that the Great Sir Pogsalot once stated:
> > >>I am saying that it is out of my control, as it is done by code I did not
> > >>write.  So the metatable check works but the subsequent rawlen() fails.
> > >
> > >   That's what I'm not understanding---the call to lua_rawlen().  To me, it
> > >sounds like a defensive programmer going overboard.  Personally, I don't
> > >believe in defensive programming as it hides bugs [1].  Trust the
> > >programmer; do NOT trust the outside world though.
> > 
> > ... and don't trust that two programmers that have never even talked to 
> > each other will always produce compatible code.
> > 
> > Consider two C extension modules written by different authors, which use 
> > the same name for `luaL_newmetatable` but for different C types. No 
> > trust in either programmer will prevent a crash when a user loads both 
> > modules.
> 
>   And that brings us back to the whole namespace issue.  Odd how that keeps
> coming up.
> 

I'm working on a run-time module loader which fetches modules over the
network on-demand[1].[2] The solution I came up with was to use a prefix
which is the base32 public key (RSA or ECDSA), or optionally a smaller
prefix of a hash of the public key. A 40-bit prefix is only 8 characters and
reasonably secure, especially if you bail on collisions, but the index
loader maps multiple prefixes (40-bits, 60-bits, 80-bits, etc).

Then in the config file, if you really want, you can associate names to the
key identifier, and annotate some trusted keys so that unprefixed modules
will be searched in the namespaces of your favorite or most trusted signers.

This obviates any need for a centralized repository or hierarchical naming
scheme. Everybody can name their packages however they want, and nobody
needs to fight over cool sounding names, because random prefixes are a great
equalizer. You can name your JSON module json.lua. End of story.

You point the package loader at various package listings published over http
and ftp. The listings are basically mappings of public keys to packages, and
each package has a human-readable name, optional versioning information,
plus the one-way hash of the actual module file and a signature of that
hash. The one-way hash of the module is the identifier used when downloading
and storing.

It also allows people to repackage stuff, publically or privately. So, for
example, I'll package as much of LuaRocks as I can (with C modules
precompiled for all the major Unix platforms) and publish it under my key
(or one of my keys). And because they're retreived by the hash of the module
itself, if multiple people repackage the same module it needn't necessarily
be stored or loaded multiple times.[3]

For long-running daemon applications (what I usually write) I'm hoping to
add a dynamic update mechanism for bug fixes security updates. Being
non-blocking, it can update itself without interrupting the application. The
intention is to be completely self-contained, without requiring any
administration on the server. In the typical case, you just require"" your
module and you're done.


1. One cool side-effect of this is that this magically solves most
dependency tracking. You don't need to document dependencies in a
configuration file because they'll be installed recursively when the module
itself is loaded. It can't do this for C modules which load other Lua
modules, but that's usually bad form anyhow.

2. I should note that it's built to do this non-blocking using modules like
cqueues (but not exclusive to cqueues). No shelling-out is allowed in the
default implementation, because that would block highly concurrent
applications and is also very non-portable, not to mention somewhat
dangerous. The only requirement is the single Lua file which implements
package management and calls retrival hooks, and my OpenSSL Lua module,
which is the only module I know of with all the necessary bindings.

3. Again, C modules pose additional problems if, e.g., they use fixed,
global identifiers, but this is surmountable.