Re: zero-copy read() into Lua string?

[Miles Bader <miles@...>]

Josh Haberman <jhaberman@gmail.com> writes:
>   char *lua_pushwstring(lua_State *L, size_t len);
>   void lua_finalizewstring(lua_State *L);
>
> (lua_finalizewstring() is there in case Lua wants to internally
> precompute a hash of the string or anything like that.)
>
> In some cases (like read()/fread()) this could save a memory copy for
> string data that is flowing from C->Lua, which seems like a big
> benefit.  Though I haven't implemented this idea to perform benchmarks,
> avoiding memory copies can improve efficiency significantly in my
> experience.

Yeah this sounds useful to me.

I read really big files (~500GB) into Lua strings (for parsing with
lpeg), and it was annoying to do this without thrashing on systems with
only 1GB physical memory.

[Currently I mmap the file, use madvise(MADV_SEQUENTIAL), and then use
lua_pushlstring; this works OK, but of course that isn't portable...]

-Miles

-- 
Bacchus, n. A convenient deity invented by the ancients as an excuse for
getting drunk.