Re: io:lines() and \0

[René Rebe <rene@...>]

On Feb 19, 2014, at 19:12 , René Rebe wrote:

On Feb 19, 2014, at 18:45 , Sean Conner wrote:

It was thus said that the Great Craig Barnes once stated:

On 19 February 2014 16:41, René Rebe <rene@exactcode.de> wrote:

Who said it was impossible? A simple loop fixing this issue was presented by

Francisco.

No one did. It was just in reply to:

Amazing how much energy goes into preventing other people to improve something

Your previous patch looks promising.

 It has a bug.  If the "line" read in is LUAL_BUFFERSIZE bytes (or longer)
in size, *and* the byte *at* LUAL_BUFFERSIZE is 0, then the routine returns
a corrupted line.

I think that case is covered, but there is indeed a bug: it returns garbage if there is no newline at the end of the file. So non-C99 platforms would only be supported with the getc loop, … :-/

For some magic reason tailing zeroes work here on Mac on Linux with my last patch, I will debug the buffers tomorrow to see why that is the case.

However, a magic free version would be to pre fill the whole buffer with newlines, to guarantee detection of the end. However, this is only more efficient than the for loop using getc with a local buffer size smaller than the default BUFFERSIZE which is apparently 8192 on my Linux system and 1024 on Mac OS X. With a local buffer of just 80 chars I get some moderate ~10% slowdown on the bible test case.

Given this IO function may not be the most performance critical code path in Lua, that may be acceptable. But your milage may vary; and so I get the feeling nobody will like it here in any case.

René

-- 
 ExactCODE GmbH, Jaegerstr. 67, DE-10117 Berlin
 http://exactcode.com | http://exactscan.com | http://ocrkit.com | http://t2-project.org | http://rene.rebe.de