Re: load() and standard iterators for reading files in 5.2

[Duncan Cross <duncan.cross@...>]

On Fri, Feb 26, 2010 at 12:11 PM, Duncan Cross <duncan.cross@gmail.com> wrote:
> On Thu, Feb 25, 2010 at 8:52 PM, Roberto Ierusalimschy
> <roberto@inf.puc-rio.br> wrote:
>>> not entirely convinced that the only reason you would ever want to use
>>> a "loadfilein" is sandboxing, but then in truth I don't have an actual
>>> counterexample to support that. Sorry for the noise, anyway.)
>>
>> Again maybe I was not very clear. The specificity of the situation was
>> not only the sandboxing, but the absence of the io lib. With the io
>> lib, you should be able to do a loadfilein like here:
>>
>>  loadin(env, io.lines(filename))
>>
>> (except that this does not work, because 'io.lines' cut the newlines
>> from the file; we intend to fix that somehow...)
>>
>> -- Roberto
>>
>
> Oh, that's a good point. I had assumed that you would need to do
> something pretty involved with coroutines etc. to write your own
> equivalent function, which might be a bit of a tall order for the
> less-experienced - I hadn't thought about using io.lines() as a
> standard iterator for load(), that is quite a powerful idea.
>
> There's a few solutions I can think of to the newlines problem:
> + Add a new extra boolean parameter to io.lines() to specify that you
> want to preserve them
> + Allow functions passed to load() to return multiple strings instead
> of just the one, and change io.lines() iterators to yield two results
> each time, the line and the new line character(s)
> + Add a new function, a generalised form of io.lines(), something like
> io.reader(filename, format) where 'format' is the same kind of thing
> as the argument(s) passed to file:read(). If you pass "*l", the
> iterator returned is identical to one created by io.lines() (and would
> have the same problem); you could instead pass a number or "*a", which
> would create an iterator that would not strip the newlines.
>
> Personally I like the third option, as it seems like a more general
> solution. Any which way, I think it is worth finding a good solution
> in time for the final release of 5.2.
>
> -Duncan
>

I just noticed I've confused the issue here by talking about load()
all the time - of course, you do not need a special iterator for
reading files with load(), because loadfile() works fine. It is
loadin() that it would be useful for.

-Duncan