Re: Pooling of strings is good

[Coroutines <coroutines@...>]

On Sat, Aug 23, 2014 at 12:11 AM, Sean Conner <sean@conman.org> wrote:

>   It doesn't bother me, because the convenience it affords is worth the
> price (in my opinion).

What convenience?  I'm saying the interface doesn't exist to make
mutable data (userdata) as accessible as immutable, pooled strings.

>   I swear, the way you are talking, it's as if you equate "userdata" with
> "Lua's internal string representation" and it jars me every time.  I have
> plenty of userdata types that have nothing to do with strings; where
> "userdata == string" just does not make any semamtic sense what-so-ever [1].

I should word this more carefully: I mean to say that userdata and
strings (the internal structures and how they are handled in Lua) are
very similar.

http://www.lua.org/source/5.2/lobject.h.html#TString

Both are basically: header + data, the size of the data that follows
is known in the header.  Strings have a hash, userdata have
environment tables -- but otherwise they are very similar.  It would
be trivial to walk the contents of either as you know where it starts
and where it ends.  In Lua you can walk/traverse strings, but you
cannot do the same with userdata.  Often it is "simpler" to just
dump/promote the userdata to a string.  I will correct myself here: I
thought lua_Buffer's were a wrapper around userdata because it would
have made sense to me to reuse data structures and the functions
defined for them.

They are not: http://www.lua.org/source/5.2/lauxlib.h.html#luaL_Buffer

> so again, I'm at a loss as to what "userdata == string" even means in that
> case.

I'm saying I want to be able to write 'cat' == io.stdout and have it
do a memcmp() of their contents (the data section after the header).

>   So really, what you are after is a buffer type userdata that can be used,
> invisibly, as an immutable string.  You could hack Lua to do that, as a
> proof-of-concept, get some feedback, do some benchmarking and possible get
> enough users to convince PUC to officially add it.

I'm not necessarily asking for a whole new defined mutable type.  I
just want the ability to use userdata where a string would be
expected.  The mutable type is already available: userdata

All I'm thinking about is things like this:

string.sub(io.stdout, 1) -- dumping the lua_Stream struct into a Lua string

Or this:

debug.getregistry()['FILE*'].__index = function (self, key) return
self[key] or string.sub(self, key) end -- and then you could: for i =
1, #io.stdout do local b = io.stdout[i] ... end

(You could traverse userdata byte-by-byte like a string)

I'd love to just be able to read the contents of a userdata from Lua
:\  I realize this breaks encapsulation, so maybe there'd have to be
some switch/option in the debug library to use the string functions on
userdata (to allow or not).

>   I use 22 userdata types at work (just counted).  It's not a small concern.

I mean to say -- how  many of those are you defining personally?  How
many are brought in from third-parties?  I don't think you'd have much
problems deciding the names for each..

Related: It would be nice to be able to change the typename at runtime
(even so that checkudata passes) -- without having to recompile the
third-party library that defines the userdata type.

>   I never said it was static.  That "buffer" there is an auto---declared on
> the stack of the socklua_recv() function [5].  No static buffer here.

My bad, I thought I read static :\  Stack overflows are fun... :p  I
was just thinking offhandedly that you could create a mmap, expose it
to Lua as a file to read and write from -- and pass it to the
networking lib as the buffer to recv into.  Hmm.  And it could be
shared depending on the threading setup -- but that's an afterthought.

>   And the way I handle multithreaded process with Lua embedded is to give
> each operating system thread its own Lua state.  Yeah, I could provide the
> lua_lock() and lua_unlock() implementations, but I'd rather avoid the loss
> of speed that a global interpreter lock imposes [6].

I really wish I could just require() in the libs I want accessible to
"ALL TEH THREADS" and share that Lua state.  Locking is just too much
of a penalty...

On another side-topic: It makes me really unhappy that
lua_lock/unlock() are macros and not dummy functions.  This is Lua's
GIL :(  You have to recompile Lua to make use of them when I'd rather
LD_PRELOAD off of the Lua my distro distributed...  c'est la vie.