Re: Proposal: smartlua

[Sean Conner <sean@...>]

  Keying off the last message from Steve Donovan, who said:

> So the question would be: what would be the preferred workflow?  The
> way I imagine it going is that a person has a nice Lua program and
> wishes to deploy it on multiple machines and hand it out to chums.

  let's go back to the message that started this:

It was thus said that the Great Stefan Reich once stated:
> smartlua should be: an executable that runs Lua code easily.

  Okay.  I'm assuming Lua 5.1 because that's what I'm working mostly with
right now.  Let's see some requirements ...

> smartlua should:
>   -run on Linux and Windows

  Okay.  That already leaves me out as I also use Solaris and MacOS.  But
I'll assume Linux == Unix in this case.

>   -be delivered as a single executable (shell script/.exe, resp.)

  Now, does this mean a "Kitchen Sink Lua" with all the modules built into
the executable so nothing else need be installed?  Or just the interpreter,
like "lua.exe" and any external modules as separate files, all bundled into
an archive like a zip or tar file?

  The former is *frighteningly" difficult to automate (having done a Kitchen
Sink Lua by hand, I can't even begin to fathom the work required to have
that done automatically), while the later appears graspable.

>   -look at the Lua source file(s) and guess what libraries are needed

  Okay, in looking through an arbitrary script, you will need to look for
the following functions:

	require()
	dofile()
	load()
	loadfile()
	loadstring()

require() is perhaps the easiest to deal with.  You'll need to parse the
module name and search for it.  Pretty easy (code to do this below) and
you'll need to (as a guess) replicate in some target directory (for
archiving) the lib/ and share/ structure.

loadfile() is a bit more work.  Best bet would be to create another
subdirectory, say, 'files', and rework the filename to be based off this
directory.  Why?  Because if the user specified something like
'/usr/local/sbin/syslog.lua' it'll be difficult to extract the file to that
location on the target system (most likely because of permission issues).  

But that brings up another issue---how the filenames are specified.  I mean,
I can see something like this being done:

	OS = io.popen("uname -a","r"):read("*a")
	if OS == 'SunOS' then
	  local = "/opt"
	else
	  local = "/usr/local"
	end
	
	x = loadfile(local .. "/Frob/run.lua")

so not only do you need to search for loadfile(), but you'll need to parse
the argument, see that it's a string concatenation, and backup to where
everything is defined to reconstruct the filename, as it were.

  I threw in load() and loadstring() as well, as that is also were code can
be introduced, and again, you'll need to do an extensive parsing job to
figure out if we need to include a file or not.  load() will be particularly
hard to handle as it takes a function that returns code to be parsed.

  I suppose one could argue that dofile(), load(), loadfile() and
loadstring() should be ignored for this purpose, but then there goes the
bullet-proof anyone can package anything up program (but "perfect" is the
enemy of "good enough"---but in this case, what is "good enough"?)

  This also brings up addtional data files.  I have one module (work
related) that reads in three large datafiles (via io.open()/file:read())
when the module is loaded.  For the Kitchen Sink Lua I had to write a Lua
script that is run from the Makefile to generate the source for the module
with the datafiles inlined, as it were; I couldn't use the module as written
because it assumed it could open the datafiles and I wanted a "no-install"
Lua interpreter.

>   -make includes be found automatically

  I'm not sure if I understand this one---do you mean the ability to find
makefiles and invoke make to build code?  This needs a clarification.

>   -download and install standard libraries if needed

  Easier said than done.  LuaRocks, for instance, has a Solaris target, but
a) it didn't work because b) Solaris isn't Linux and c) we use the Solaris C
compiler at work, not GCC, a LuaRocks assumption that isn't always true (I
don't blame the LuaRocks author(s) for this, they probably don't have access
to corporate run Solaris boxes).  

  But leaving Solaris aside and going to Linux, I have two (2) Linux systems
that are now incapable for downloading and installing "standard libraries"
because they are older than twenty minutes and thus, horribly outdated and
no longer supported and I should upgrade to the latest and greatest on a
continuous 24-hour stream and well ... @#@#$@! that!  What I have works and
if I need something, I can very well download the source and compile (I work
on the principle of "if it ain't broke, don't fix it").  Also, what
distribution of Linux?  Debian?  Ubuntu?  RedHat?  CentOS?  Slackware? 
Gentoo?  

  If I'm sounding too negative, I'm just pointing out the issues *I'm*
seeing in looking over this for five minutes.  I think it's better to hash
out some of these issues now, possibly define some boundary conditions and
what the caveats exist for people who might want to use such a program (this
only catches modules explicitely listed in require(); any additional files
required by the code are beyond the scope of the program, etc, etc.).

> First step to realizing smartlua: Let's collect example scripts you run
> all day.

  Okay, here's a [sanitized] module I use at work:

local util = require "util"

module("person",package.seeall)

-- ***********************************************************************

local function load_file(filename)
  local file = io.open(filename,"r")
  local list = {}
  
  for line in file:lines() do
    table.insert(list,line)
  end
  file:close()
  return list
end

DATADIR = util.data_path("data") 
if DATADIR == nil then 
  error("can't find data directory")
end

MALE   = load_file(DATADIR .. "/male.txt")
FEMALE = load_file(DATADIR .. "/female.txt")
FAMILY = load_file(DATADIR .. "/family.txt")
SEX    = { MALE , FEMALE }

-- ***********************************************************************

local function genname(sex)
  return string.format("%s %s",sex[math.random(#sex)],FAMILY[math.random(#FAMILY)])
end

-- ************************************************************************

function new()
  local person  = {}
  
  person.sex    = SEX[math.random(2)]
  person.first  = person.sex[math.random(#person.sex)]
  person.last   = FAMILY[math.random(#FAMILY)]
  person.name   = string.format("%s %s",person.first,person.last)
  return person
end

-- *********************************************************************

This is the module I talked about above.  Yes, I need to generate random
names as part of my job (testing).  And yes, this is a sample script I run
every day.

  -spc (Another fun one to contemplate:  my JSON parser at
	https://github.com/spc476/LPeg-Parsers/blob/master/json.lua, and
	yes, that IS embedded C code you see in there ... )

[1]	A 32M Lua script anyone?
	http://www.lua.org/bugs.html#5.1.4-6