Re: Apache Portable Runtime

[Rici Lake <lua@...>]

Peter Odding wrote:
> Rici Lake:
>  > The point of using a language like Lua, in my opinion, is: ease of use,
>  > avoidance of bugs, and rapidity of development. Fast execution is way 
> down the
>  > list (my list, anyway). So a good interface, in my view, should 
> promote those
>  > virtues, and only then try to do so as efficiently as possible.
> I agree that a good interface is more important than speed. However I'm 
> also considering that one can create a high level interface from a low 
> level one but not (always) vice versa (see BDUF above :-).

That's true, but that doesn't necessarily mean trying to implement
all possible micro-optimizations, which as you say below only
complicates the man pages. :)

> Thanks for the performance comparison. I should know not to optimize 
> before actual benchmarking; I get carried away sometimes :P. When I see 
> hardware -> operating system -> apache portable runtime -> lua-apr -> 
> lua, I worry about the many layers of abstraction. For example a lot of 
> APR functions use memory pools for strings but Lua also uses interned 
> strings, which means every string exists twice in memory. I guess this 
> is the price to pay for portability and safe API's.

I've since done benchmarks on other OS's and filesystems, and it appears
that the FreeBSD system I was using initially has a particularly slow
stat(). However, I think the point stands; even on systems with a fast
stat() library function, table creation is not really the major issue.

>  > Mike's suggestion is not bad, but it will prove to be awkward when 
> you want to
>  > save the entire stat structure, in which case a table will prove to 
> have been
>  > better.
> Mike's suggestion contained three calls of which the first returns a 
> table with string keys. Did you miss that one or are you referring to 
> something else?

I did miss that.
> I've rewritten the stat binding to follow the interface Mike suggested. 
> When multiple values are returned unavailable values are nil to preserve 
> the order. I've also added the function apr.lstat, which doesn't follow 
> symlinks (apr.stat with a boolean true upvalue to distinguish the calls).

That's a good idea.

> Roberto Ierusalimschy:
>  > Maybe you may want to add yet another option:
>  > Explicit table given: return that table with all (named) properties.
>  > ...
> Following your suggestion I added this option. On second thought however 
> the interface Mike suggested seems flexible enough so I've removed this 
> option again. Otherwise the manual entry would get rather long and 
> confusing.

I did some benchmarking on that as well. There is an argument for
recycling tables -- it cuts down on memory allocations -- but on
the systems I have available to me, the cost of the memory allocations
seems to be quite small. If you're really concerned about speed, the
biggest win seems to be keeping the table keys in Lua, rather than
lua_pushstring()'ing them on every table creation. I've got some
sample code which is roughly based on lhf's lposix but uses
precreated strings, and it is about 25% faster.

One of the things I discovered in doing the benchmarks is that
the table recycling interface can actually be slower if the
table you're creating has optional keys. In the case of a
new table, you can create the table to the correct size and
only populate it with keys which actually exist; with a
recycled table, you have to set the optional keys explicitly
to nil. (In the sample code I wrote, I used optional keys for
setgid/setuid/sticky bits as well as rdev.)

I generally dislike table recycling because it leads to subtle
bugs; in any event, given an interface which produces either
a new table or a multiple return of specified keys, it is
easy enough to write a Lua function to do it. For example,
a file monitoring object might look something like this:
(untested)

do local monitor = {}; monitor.__index = monitor -- meta
  -- just updates the data
  function monitor:update(filename)
    filename = filename or self.filename
    self.filename = filename
    if filename then
      local ino, dev, size, mtime =
        apr.stat(filename, "ino", "dev", "size", "mtime")
      if ino == nil then
        self.valid = false
        self.lasterror = dev
      else
        self.ino = ino; self.dev = dev; self.size = size;
        self.mtime = mtime; self.valid = true
      end
    end
    return self
  end

  -- check to see if anything has changed
  function monitor:changed()
    local valid, ino, dev, size, mtime =
      self.valid, self.ino, self.dev, self.size, self.mtime
    self:update()
    return not(valid and self.valid
               and ino == self.ino and dev == self.dev
               and size == self.size and mtime == self.mtime)
  end

  function Monitor(filename)
    return setmetatable({}, monitor):update(filename)
  end
end