lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Now you mention it, here's a related feature request for the next LuaSocket. For line-by-line reading, allow an additional numeric parameter for the max no of bytes to read. Return at EOL or after max bytes, whichever comes first.

I have actually thought about implementing this one. I even wrote partial support for it in the form of a "*<n>l" or "*<n>a" patterns, where <n> gives the maximum size. The functions would return "truncated" if the maximum size was reached before the end of the pattern.

But then I noticed that any serious server would be doing non-blocking I/O anyways (which already returns partial results).

Well, you might be writing a "serious" server using blocking I/O and multiple threads, e.g. with LuaTask, instead of using non-blocking I/O and coroutines, e.g. with COPAS. (If very high concurrency is not an issue, both approaches are programmatically appropriate -- different strokes for different folks.)

And the other reason, even with non-blocking I/O, is that it can still be considered handy to limit the maximum amount of data you'll have to handle in one go. (You might get a lot of already-buffered-elsewhere data back before blocking, after all.)

Another featurette which could be considered as a generic version of your patch is to have a single char to specify the EOL char (you would just use a zero byte).

I try to cover most of the use cases, but I am usually avoid introducing to too much complexity into the library. I know, each request seems simple, but once they pile up, things become untreatable. Are there many protocols that behave like this? Examples would be helpful.

SMTP and HTTP spring to mind. My point is that as things stand, you can't use the mightily convenient "*l" when writing, say, an SMTP server which implements (or at least allows as an option) _strict_ compliance with RFC2821. I believe the same is true for HTTP (many servers allow lines to end just LF, but strictly speaking, CRLF is required). Numerous other line-oriented protocols specify CRLF, if I recall correctly, and actually disallow CR on its own (a condition you cannot test for with "*l" because it blindly eats CRs) anywhere, or LF on its own.

I just think that completeness requires that you be able to reconstruct the input data after receiving it, and it seems a pity that you can't do that if you want to use "*l" -- you have to implement your own buffering in Lua on top of the buffering already implemented below.

The change I had in mind is _very_ simple. For example, if you use "=x" to mean "read a line of data terminated by x", then this should do the trick.
There are three lines of executable code changed and just one added:

--- buffer.old	2007-07-21 15:45:41.000000000 +1000
+++ buffer.c	2007-07-21 15:53:55.000000000 +1000
@@ -15,3 +15,3 @@
 static int recvraw(p_buffer buf, size_t wanted, luaL_Buffer *b);
-static int recvline(p_buffer buf, luaL_Buffer *b);
+static int recvline(p_buffer buf, luaL_Buffer *b, char eol, int eatcr);
 static int recvall(p_buffer buf, luaL_Buffer *b);
@@ -121,3 +121,4 @@
         const char *p= luaL_optstring(L, 2, "*l");
-        if (p[0] == '*' && p[1] == 'l') err = recvline(buf, &b);
+        if (p[0] == '*' && p[1] == 'l') err = recvline(buf, &b, '\n', 1);
+        else if (p[0] == '=') err = recvline(buf, &b, p[1], 0);
else if (p[0] == '*' && p[1] == 'a') err = recvall(buf, &b); @@ -216,6 +217,6 @@
-* Reads a line terminated by a CR LF pair or just by a LF. The CR and LF -* are not returned by the function and are discarded from the buffer +* Reads a line terminated by an eol char, optionally "eating" CRs. The +* eol char is not returned by the function and is discarded from the buffer.
-static int recvline(p_buffer buf, luaL_Buffer *b) {
+static int recvline(p_buffer buf, luaL_Buffer *b, char eol, int eatcr) {
     int err = IO_DONE;
@@ -225,5 +226,5 @@
         pos = 0;
-        while (pos < count && data[pos] != '\n') {
+        while (pos < count && data[pos] != eol) {
             /* we ignore all \r's */
-            if (data[pos] != '\r') luaL_putchar(b, data[pos]);
+            if (eatcr == 0 || data[pos] != '\r') luaL_putchar(b, data[pos]);