Re: Pooling of strings is good

[Coroutines <coroutines@...>]

On Fri, Aug 22, 2014 at 2:13 PM, Sean Conner <sean@conman.org> wrote:

>> I play with networking.

So then you know what a pain it is to not be able to represent a
simple ring buffer, or provide DMA access to the NIC's ring buffer
hosted through userdata to Lua -- without having to copy it out and
promote it to a string?  I wish userdata could be used in the same
place a string is expected.  I would love an "invisible" memcmp() if
it's a userdata-string comparison (==) or to be able to directly use
the string.*() functions on them.  Strings are sealed userdata -- if
you know you need a fixed buffer whose contents are changed quite
often then I wish userdata could conform to this use.  Generally
duplicating them into a string is okay because we don't handle very
large data, but lots of frequent reallocation frustrates me.  This
stuff doesn't need to be in Lua's string pool for memory or comparison
efficiency :p  All I really need is read access to it -- and I don't
want to have to rewrite what's in the string library for userdata.

>   But now you are exposing what has been an implementation detail.  The
> tname just has to be unique, it doesn't have to have any meaning (so I guess
> the days of #define MY_TYPE "\200\201\202\203\344\377\300" are over) and we
> run headlong into the namespace problem we (potentially) have with modules,
> but now with types.

I think -- like module names -- people will have to put some thought
into their typenames.  I would not expect people to pull in libraries
introducing a lot of userdata types anyway, I feel like it would be a
small concern.

>   Other unintended consequences---why did I create a new function typeof()?
> Why not extend type() to return a second value?  Because there might be
> code that expects type() to only return a single value (calling type()
> within the definition of an array)?

Disagreement here, I would avoid a typeof() and just add the 2nd
return to type() -- or have a type() and rawtype().  Doesn't really
matter ~

> Hmmm ... this is bringing up an interesting problem---do we need to check
> for mutable strings (buffers?) and immutable strings?  What happens to C
> code that does:
>
>         const char *name = lua_tostring(L,idx);

I imagine nothing changes, as you are referencing const char.. :p

>         char buffer[65536uL];
>         bytes = recvfrom(sock->fh,buffer,sizeof(buffer),0,&remaddr->sa,&remsize);
>         lua_pushlstring(L,buffer,bytes);

The only problem I have with this is while I know of no foolproof way
to multithread an embedded Lua process, it seems like a bad idea to
have a "shared" static buffer in a library? :>

>   Okay, nothing stopping anybody from doing that.  But I suspect only a
> small subset of the standard Lua library could be done in pure Lua.

Mostly I just thought it would be clearer to see how args are accepted
and parsed and it would be easier to notice issues/bugs.  The other
bonus would be it'd be easier for upstream and the community to
prototype new functions to add to the string or table or whatever
library.  The standard libraries themselves could be released as less
of a 'standard' and more of a 'go add to it' example.  I think Lua
would grow more quickly because of it -- and in ways the community
could regard and come to a consensus on?

>
>> PS: String are userdata :(  I don't want to treat them like a special
>> type when they *are* just userdata with added magic.  We could do it
>> in Lua.  userdata used for a buffer type would be much more applicable
>> ~everywhere~, I feel like I have to turn to C to make the best of it
>> and that frustrates me.  I like working only in Lua if I can.
>
>   -spc (By the way, how much C programming do you do?)

I'd like to think I'm fairly knowledgeable of C -- I've been at it for
7 years now but that is in no way a good measurement of if someone is
"capable" in C.  I mostly use it for making bindings today if I can
prototype in Lua or JS faster.  I've made bindings for BSD sockets,
libev, and libevent in Lua.  I've been toying with nodejs in
Javascript so there is no real need...  Before all this I used to
write code for a certain IRC daemon.