Re: [ANN] LunarCGI finally supports fileuploads!

[Miles Bader <miles@...>]

HyperHacker <hyperhacker@gmail.com> writes:
> So uploading a 100MB file will consume 100MB of RAM? There's a good
> reason PHP just gives you a handle to a temporary file and a way to
> move it.

It would be cool if there were a way to have a Lua string whose contents
were actually an mmap'd buffer (or any externally-allocated buffer), but
I can't see an easy way to do this -- the Lua string object actually
stores the string contents following a small header...

Then you could easily process "strings" without copying them into
memory.

When I needed to process large files (~1-2GB) as Lua strings, I mmapped
them, marked the memory as "MADV_SEQUENTIAL", and made Lua strings from
them using lua_pushlstring.  This incurs the copy overhead and uses
anonymous memory, but at least it's a single straight copy, and I
haven't have too many thrashing problems doing this (whereas using
io.read"*a" is a complete disaster for these files).  Of course things
would get worse at some point...

-Miles

-- 
Bigot, n. One who is obstinately and zealously attached to an opinion that
you do not entertain.