Re: io:lines() and \0

[Francisco Olarte <folarte@...>]

Hi:

On Fri, Feb 21, 2014 at 11:59 PM, Sean Conner <sean@conman.org> wrote:
>   I think it's more an investigation of "how do other languages handle
> this?" than trying to look for worse.
>   C, using fgets(), would print:
> hellohellohellohellohello
>   but only because of the way C treats NUL in strings.

At least C would get the line count half right, but:

folarte@paqueton:~/tmp$ cat lines.lua
n=1;
for line in io.input():lines('*L') do
    print(string.format("%d: '%s'",n,line))
    n=n+1
end
folarte@paqueton:~/tmp$ (for a in  1 2 3  ; do echo -ne
"hello\0world\n" ; done) | lua lines.lua
1: 'hellohellohello'
folarte@paqueton:~/tmp$

Another difference is that every C programmer ( but not everyone who
is able to write and compile C code ) knows nuls are special in C and
does not use fgets on untrusted files, as C has no way to signal you
these problems ( either in fgets or by telling you a file is not text
). This I why I always used getc ( not fgetc, getc began to be much
faster long time ago, and know I know it I will use *_unlocked until I
find the fscking switch to tell the library 'I'm single threaded,
gimem the nlocked by default  ) when possible, and some times fread (
no getline then ). I also knew opening text was no magic bullet, and
if you opened a binary file in text mode you got strange results.

Since them times have changed, and we've got new languages which are
better for text manipulation. Many of them have nul-safe strings, and
the one which have them normally have no problems with embeded nuls or
any other thing you throw at them, like python which has been tested
above, or perl:

folarte@paqueton:~/tmp$ (for a in  1 2 3  ; do echo -ne
"hello\0world\n" ; done) | perl -ne 'print "$. : $_"'
1 : helloworld
2 : helloworld
3 : helloworld

And even not-null safe bash gets half right:
folarte@paqueton:~/tmp$ (for a in  1 2 3  ; do echo -ne
"hello\0world\n" ; done) | ( while read LINE; do let ++NUM ; echo
$NUM: $LINE; done )
1: hello
2: hello
3: hello

Regards.

Francisco Olarte.