zero-copy readonly strings from C to Lua?

[Bennett Todd <bet@...>]

I'm noodling out a design for an email manipulation tool; I'm sorta
fantasizing that I could start out by knocking out a
procmail/maildrop killer, and grow it into a general purpose
filtering/rewriting engine for all sorts of contexts. This is
something I've kinda had in the back of my head for years, haven't
yet gotten around to actually doing, may not this time, but....

One thing I'd want it to do is be as absolutely high-performance as
possible. In pursuit of that, I was thinking that the ideal way to
manage the messages would be to do the basic I/O processing purely
on the C side, and only expose bits of the message info --- envelope
data, header info, body structure, etc. --- up in to Lua on demand.
So a pure local delivery agent with no filter --- read message from
stdin, envelope recipient on cmdline, write to that recipient's
~/Maildir/ --- could be done with an absolute minimum of data
manipulation, minimum of syscalls, no extra data copies, and none of
the message ever exposed to Lua. Simple filtering would require
parsing the headers and exposing them to Lua; more elaborate
processing would get into exposing part or even all of the message
body to Lua.

As I continued to think about it, it seemed to me like Lua's string
representation would be ideal for letting Lua point right into the
string buffer where the [parts of] the message are being held by C.

But as I look in the Lua reference manual 4.0, I see in section 5.5:

        In particular, lua_pushlstring and lua_pushstring make an
        internal copy of the given string.

Is there some other documented, portable (across Lua versions) way
for C to offer a string to Lua?

Is it architecturally impossible to allow straight normal Lua code
to directly see as strings some bytes that a supporting C library
holds in its buffers, without any copying?

I can understand that it would be hard to interact with GC and
string consolidation and so forth, but I'd think if I made a
guarantee to never again touch those bytes for the life of the given
lua_State, Lua should be able to accept some memory from C without
having to always and invariably copy it. No?

-Bennett

Attachment: pgpkILpVNhVFv.pgp
Description: PGP signature