lua_Stream in Lua 5.2 and Lua 5.3

[Jan Behrens <jbe-lua-l@...>]

Hi,

I have been working on a TCP application server framework for Lua 5.2.
Instead of creating my own I/O stream object, I'm using the luaL_Stream
struct with the LUA_FILEHANDLE metatable in the following manner:

// int fd;  file descriptor of the network socket
luaL_Stream *stream;
stream = lua_newuserdata(L, sizeof(luaL_Stream));
stream->f = fdopen(clonedfd, "b");
stream->closef = my_close_stream_func;
luaL_newmetatable(L, LUA_FILEHANDLE);
lua_setmetatable(L, -2);

I have noticed that while this seems to work fine with Lua 5.2, it has
not been part of its documentation but only been mentioned in the
documentation of the upcoming Lua 5.3:

http://www.lua.org/work/doc/manual.html#luaL_Stream

Question:
Is there any reason not to use this mechanism with Lua 5.2 already?


I really appreciate this interface to pass FILE streams to the Lua
world. However, programming a network server, I have been missing one
particular function of Lua's I/O library: reading until the end of line
or until a limit of bytes has been reached, whatever comes first.

This is an important feature when dealing with network sockets.
Implementing this in pure Lua is either slow (requires reading one byte
at a time) or complicated (requires a secondary buffer as you cannot
"unread" anything that has been read). However, it is possible to
implement it in C as follows:

static int readuntil(lua_State *L) {
  luaL_Stream *stream;
  const char *terminator;
  size_t terminatorlen;
  luaL_Buffer buf;
  lua_Integer maxlen;
  int byte;
  stream = luaL_checkudata(L, 1, LUA_FILEHANDLE);
  terminator = luaL_checklstring(L, 2, &terminatorlen);
  luaL_argcheck(L, terminatorlen == 1, 2, "single byte expected");
  maxlen = luaL_optinteger(L, 3, 0);
  if (!stream->closef) luaL_error(L, "attempt to use a closed file");
  luaL_buffinit(L, &buf);
  if (!maxlen) maxlen = -1;
  while (maxlen > 0 ? maxlen-- : maxlen) {
    byte = fgetc(stream->f);
    if (byte == EOF) {
      if (ferror(stream->f)) abort();  // or any other error handler
      break;
    }
    luaL_addchar(&buf, byte);
    if (byte == *terminator) break;
  }
  luaL_pushresult(&buf);
  if (!lua_rawlen(L, -1)) lua_pushnil(L);
  return 1;
}

And then registering such a method, for example, as follows:

luaL_newmetatable(L, LUA_FILEHANDLE);
lua_getfield(L, -1, "__index");
lua_pushcfunction(L, readuntil);
lua_setfield(L, -2, "readuntil");
lua_pop(L, 2);

Which then allows you to read a line over the network in the following manner:

-- let 'socket' be a Lua file handle
local line = socket:readuntil("\n", 8192)
if not line then return end
local method, url, proto = line:match("^([^ \t\r]+)[ \t]+([^ \t\r]+)[ \t]*([^ \t\r]*)[ \t]*\r?\n$")

Using socket:read("*L") instead might consume a huge amount of memory
if a (malicious) peer does never send a new-line character.


While my approach of extending Lua's file objects with a new method
works just fine, I would like to propose to implement a similar
functionality within Lua itself, e.g. by extending the file:read method
(and io.read function), or by creating a new method/function that
accepts an additional parameter that limits the number of bytes read.


Kind Regards
Jan Behrens



-- 
Public Software Group e. V.
Johannisstr. 12, 10117 Berlin, Germany

www.public-software-group.org
vorstand at public-software-group.org

eingetragen in das Vereinregister
des Amtsgerichtes Charlottenburg
Registernummer: VR 28873 B

Vorstände (einzelvertretungsberechtigt):
Jan Behrens
Axel Kistner
Andreas Nitsche
Björn Swierczek