Re: Lua Socket

[Duck <duck@...>]

> > Now you mention it, here's a related feature request for the next 
> > LuaSocket. For line-by-line reading, allow an additional numeric 
> > parameter for the max no of bytes to read. Return at EOL or after max 
> > bytes, whichever comes first.

> I have actually thought about implementing this one. I even wrote partial 
> support for it in the form of a "*<n>l" or "*<n>a" patterns, where <n> 
> gives the maximum size. The functions would return "truncated" if the 
> maximum size was reached before the end of the pattern.

> But then I noticed that any serious server would be doing non-blocking 
> I/O anyways (which already returns partial results).

Well, you might be writing a "serious" server using blocking I/O and 
multiple threads, e.g. with LuaTask, instead of using non-blocking I/O and 
coroutines, e.g. with COPAS. (If very high concurrency is not an issue, 
both approaches are programmatically appropriate -- different strokes for 
different folks.)

And the other reason, even with non-blocking I/O, is that it can still be 
considered handy to limit the maximum amount of data you'll have to handle 
in one go. (You might get a lot of already-buffered-elsewhere data back 
before blocking, after all.)

> > Another featurette which could be considered as a generic version of 
> > your patch is to have a single char to specify the EOL char (you would 
> > just use a zero byte).

> I try to cover most of the use cases, but I am usually avoid introducing 
> to too much complexity into the library. I know, each request seems 
> simple, but once they pile up, things become untreatable. Are there many 
> protocols that behave like this? Examples would be helpful.

SMTP and HTTP spring to mind. My point is that as things stand, you can't 
use the mightily convenient "*l" when writing, say, an SMTP server which 
implements (or at least allows as an option) _strict_ compliance with 
RFC2821. I believe the same is true for HTTP (many servers allow lines to 
end just LF, but strictly speaking, CRLF is required). Numerous other 
line-oriented protocols specify CRLF, if I recall correctly, and actually
disallow CR on its own (a condition you cannot test for with "*l" because 
it blindly eats CRs) anywhere, or LF on its own.

I just think that completeness requires that you be able to reconstruct 
the input data after receiving it, and it seems a pity that you can't do 
that if you want to use "*l" -- you have to implement your own buffering 
in Lua on top of the buffering already implemented below.

The change I had in mind is _very_ simple. For example, if you use "=x" to 
mean "read a line of data terminated by x", then this should do the trick.
There are three lines of executable code changed and just one added:

--- buffer.old	2007-07-21 15:45:41.000000000 +1000
+++ buffer.c	2007-07-21 15:53:55.000000000 +1000
@@ -15,3 +15,3 @@
 static int recvraw(p_buffer buf, size_t wanted, luaL_Buffer *b);
-static int recvline(p_buffer buf, luaL_Buffer *b);
+static int recvline(p_buffer buf, luaL_Buffer *b, char eol, int eatcr);
 static int recvall(p_buffer buf, luaL_Buffer *b);
@@ -121,3 +121,4 @@
         const char *p= luaL_optstring(L, 2, "*l");
-        if (p[0] == '*' && p[1] == 'l') err = recvline(buf, &b);
+        if (p[0] == '*' && p[1] == 'l') err = recvline(buf, &b, '\n', 1);
+        else if (p[0] == '=') err = recvline(buf, &b, p[1], 0);
         else if (p[0] == '*' && p[1] == 'a') err = recvall(buf, &b); 
@@ -216,6 +217,6 @@
 /*-------------------------------------------------------------------------*\
-* Reads a line terminated by a CR LF pair or just by a LF. The CR and LF 
-* are not returned by the function and are discarded from the buffer
+* Reads a line terminated by an eol char, optionally "eating" CRs. The 
+* eol char is not returned by the function and is discarded from the buffer.
 \*-------------------------------------------------------------------------*/
-static int recvline(p_buffer buf, luaL_Buffer *b) {
+static int recvline(p_buffer buf, luaL_Buffer *b, char eol, int eatcr) {
     int err = IO_DONE;
@@ -225,5 +226,5 @@
         pos = 0;
-        while (pos < count && data[pos] != '\n') {
+        while (pos < count && data[pos] != eol) {
             /* we ignore all \r's */
-            if (data[pos] != '\r') luaL_putchar(b, data[pos]);
+            if (eatcr == 0 || data[pos] != '\r') luaL_putchar(b, data[pos]);
             pos++;