Re: Init system in Lua

[Sean Conner <sean@...>]

It was thus said that the Great Natanael Copa once stated:
> Hi,
> 
> So we (Alpine Linux) are playing with the idea of having an init
> system with lua for Linux. The idea is that instead of writing start
> up scripts in shell you write them in lua. I think this makes much
> sense because:
> * you avoid many forks = speed up
> * you can have scripts instead of compiled C stuff
> * you have a nice language which gives you arrays/hash tables etc for
> a low price (half size of bash)
> * lots smaller than glib - you can have a very small base system
> 
> This means that if we want have the possibility to have /usr on
> separate partition we must move Lua out from /usr.
> 
> This is easily done with /usr/bin/lua and /usr/lib/liblua*. My
> question now is, what about pluggable modules? /usr/lib/lua can move
> to /lib/lua but what about /usr/share/lua? I don't like the idea of
> moving it to /share/lua. Suggestions?
> 
> Since this means we will replace lots of shell scripts with lua, are
> there any lua modules for typical shell things? like mv, cp, ps, kill
> etc.
>
> Does anyone know similar projects? (bsdlua is one)

  I've played around a bit with a init system based on Lua, as well as a
inetd-type replacement.  As far as the init stuff goes, I worked out the
dependency graph for startup scripts (but without actually launching any of
the daemons).  For instance, I have the following "startup script" for
syslog:

module("syslog",package.seeall)
depends(_M,"iptables","ntpd")

description = [[
Syslog is the facility by which many daemons use to log messages to various
system log files.  It is a good idea to always run syslog.]]

function start()
  return true
end

function stop()
end

function restart()
end

function reload()
end

function status()
end

You can see that I made syslog dependend upon iptables and ntpd.  When I
"startup" the system:

(this is the dependency chain, going right to left---you can see syslog
under both iptables and ntpd)

microcode_ctl
        sysstat
                cpuspeed
                        network
                                iptables
                                        ntpd
                                                atd
                                                crond
                                                syslog
                                                        apcupsd
                                                        apmd
                                        xfs
                                        apcupsd
                                        syslog
                                                apcupsd
                                                apmd
                                        postfix
                                        sshd
                                        named
                                        httpd
                                        mysql
                                                httpd
                        iiim
                        acpid
                        smartd
                        rawdevices
                        irqbalance

(this is the order the scripts are "started".  The scripts at each level of
the dependency chain can be started at the same time---here, network, iiim,
acpid, smartd, rawdevice and irqbalance can be run at the same time)

microcode_ctl                  OK
sysstat                        OK
cpuspeed                       OK
network                        OK
iiim                           OK
acpid                          OK
smartd                         OK
rawdevices                     OK
irqbalance                     OK
iptables                       OK
ntpd                           OK
xfs                            OK
postfix                        OK
sshd                           OK
named                          OK
mysql                          OK
atd                            OK
crond                          OK
syslog                         OK
apcupsd                        OK
apmd                           OK
httpd                          OK

  This was done pretty much on a strict translation of my system's rc
scripts.  And this is as far as I got before others thing grabbed my
attention.

  The inetd-type replacement is interesting.  It's more of a daemon
launcher/monitor type program, and less like init (no dependency tracking is
done---all daemons here are launched).  This code manages the process
envinronment for the daemon, meaning that daemons can be much simpler.  By
default, the following environment is set for each daemon:

	_DEFAULT = 
	{
	  user   = "nobody",
	  group  = "nobody",
	  cwd    = "/",
	  umask  = "---rwxrwx",
	  arg    = {},
	  env    = {},
	  stdin  = file("/dev/null","r"),
	  stdout = file("/dev/null","w"),
	  stderr = file("/dev/null","w"),
	  fd     = {},
	  limits = {}
	}

  Unless overrided, daemons will be started as "nobody", with a current
working directory of "/", no argments, no environment variables, and stdin,
stdout, stderr pointed to /dev/null, no system limits, and I'll get to the
fd table in a bit.  So, a stupid example:

	stupid = 
	{
	  cwd    = "/tmp",
	  exec   = "/home/spc/source/daemon/misc/stupid", 
	  env    = { LUA_CPATH = ";;/home/spc/source/daemon/misc/?.so" },
	  stderr = dgram("/dev/log",'connect'),
	    
	  limits = 
	  {
	    stack  = '64k',
	    cpu    = '5m',
	    core   = 0,
	    nofile = 4
	  }
	}

  Here, this starts up as nobody, with only LUA_CPATH defined and stderr
pointing to /dev/log (the local socket syslog uses, so anything printed to
stderr by this service will be automatically logged to syslog), and various
system limits (kill the process after 5 minutes of CPU time).

And for a real service:

	tftp =
	{
	  user  = "tftp",
	  group = "tftp",
	  umask = "----w--w-",
	  cwd   = "/tftpboot",
	  exec  = "/home/spc/source/daemon/tftp/src/tftpd.lua",
	  env   = 
	  { 
	    LUA_CPATH = ";;/home/spc/source/daemon/tftp/src/?.so" ,
	    LUA_PATH  = ";;/home/spc/source/daemon/tftp/src/?.lua"
	  },
	  fd    = 
	  {
	    udp("0.0.0.0",'tftp')
	  }
	}

  And here we see the fd table being used.  The inetd-type program (which
will typically run as root, if that wasn't apparent yet) will open up the
files, sockets, etc, listed in fd, and when it starts the process, the next
available file descripter after stdin, stdout, and stderr will be this list. 
So, for the TFP daemon here, descriptor 3 will be the UDP socket.  This
means that the TFP daemon no longer has to include code to daemonize itself,
but it doesn't need to concern itself with opening up sockets or files,
umask, or even changing its current working directory.

  Another example service:

	qotd =
	{
	  exec  = "/home/spc/source/daemon/misc/qotd",
	  fd = 
	  {
	    udp("0.0.0.0",ports['qotd']),
	    tcp("0.0.0.0",ports['qotd']),
	    file("/home/spc/LINUS/quotes/quotes.txt","r"),
	    file("/home/spc/.quotes.index","r")
	  }    
	}

  This daemon implements RFC-865, and again, it doesn't need to concern
itself with opening up the sockets, or the actual quotes database.  In fact,
the quotes database could be set up owned by root, with only read
priviledges by root (once opened by root, the file descriptor will be
inherited by the child process, even if its own UID/GID change).  The actual
code that starts up a service:

	function run_service(service)
	  fsys.umask(service.umask)
	  fsys.chdir(service.cwd)
	  fsys.dup(service.stdin,0)
	  fsys.dup(service.stdout,1)
	  fsys.dup(service.stderr,2)
	  
	  for i = 1 , #service.fd do
	    fsys.dup(service.fd[i],i + 2)
	  end
	  closefh(#service.fd + 3)
	
	  if proc.getuid() == 0 then
	    proc.setgid(
	        unix.groups[service.group].gid,
	        unix.groups[service.group].gid,
	        unix.groups[service.group].gid
	    )
	  
	    proc.setuid(
	        unix.users[service.user].uid,
	        unix.users[service.user].uid,
	        unix.users[service.user].uid
	    )
	  end
	
	  for resource,value in pairs(service.limits) do
	    proc.limits.soft[resource] = value
	    proc.limits.hard[resource] = value
	  end
	  
	  proc.exit(proc.exec(service.exec,service.arg,service.env))
	end
	
	function start_service(name,data)
	  if data.status == 'stop' then 
	    syslog.log('notice',"Not running " .. name)
	    return 
	  end
	  
	  local child = proc.fork()
  
	  if child > 0 then
	    syslog.log('info',"Starting " .. name)
	    pids[child] = name
	
	  elseif child == 0 then
	    syslog.close()
	    run_service(data)
	  end
	end

(fsys.*, unix.* and proc.* are custom modules I wrote---pretty much think of
these as being similar in functionality to lposix)

  This code is written and works.  If a daemon exits, it will automatically
be restarted (normally nice, but if the daemon is buggy and crashes
immediately, you'll end up with said daemon continuously crashing as it's
restarted over and over again).  Many existing daemons can be run from this
service, like named and ntpd, but others, like postfix and Apache, don't fit
in this model at all.

  Ideally, a control channel would be used between the inetd-type program
and the daemons (possibly over stdin/stdout?) so that commands can be given
to the inetd-type program to manage the daemons, but that requires daemons
written to support such a channel (and modifications to the inetd-type
program as well); it's something I'd like to add, but don't have a pressing
need to yet.

  If interested, I can make this code available.

  -spc (I also have a syslogd replacement written in Lua [1] ... )

[1]	http://www.conman.org/software/syslogintr/