Re: Pooling of strings is good

[William Ahern <william@...>]

On Wed, Aug 27, 2014 at 02:30:05PM -0700, Coroutines wrote:
<snip>
> I'm thinking about making a library to expose userdata to Lua as file
> handles. You'd have userdata exist in one spot in Lua's memory and then
> you'd create the file handle which references that userdata with this
> library. This isn't doing what I'm talking about in this thread, but I
> thought it would be cool. Basically I'd use shm_open() on Linux to get an
> fd, and on Windows it'd be CreateFileMapping() (which returns a HANDLE
> that you can feed to open_osfhandle() to get an fd again). Would be kinda
> cool to operate on userdata as if it were a file or :read() strings from
> them
<snip>
> Windows: CreateFileMapping()
> Mac OS X: shm_open()
> Linux: shm_open()

I don't think shm_open provides the functionality you want because you won't
be able to wrap arbitrary blocks of memory. You'd have to first copy your
data into the shm space. shm_open + fdopen is really no different than
simply calling io.tmpfile.

POSIX has fmemopen, which is much closer to what you want. However, fmemopen
isn't very widely supported yet (so it's not portable in any practical
sense) and might be too limited for what you're trying to do, anyhow.

Even better than fmemopen are two much more flexible interfaces: 1) GNU
fopencookie, available on Linux/glibc; and 2) BSD funopen, available on OS X
and other BSDs. Both of these allow you to instantiate a real FILE handle
with user-specified function pointers for read, write, seek, and close
operations. And all those function pointers will be passed a user-specified
context object, which can be a structure with all the relevant state you
need to wrap arbitrary application objects.

funopen and fopencookie are nearly identical. It's trivial to write a thin
wrapper over both of them, and you'll be much more portable in practice than
using POSIX fmemopen or open_memstream. (OS X doesn't have fmemopen,
anyhow.)