Re: Alternative string concat (was "out of memory ")

[Philippe Lhoste <PhiLho@...>]

On lun. 07 oct. 2002 22:09:40, "Thomas Wrensch" <twrensch@uop.edu> wrote:

> My suggestion would be to write a function "join" or some such that does
> just what you want: it takes some number of strings as arguments and
> returns a string equal to all those strings concatenated together.
> 
> Thus, rather than doing this:
> 
> text = "You peek carefully around the corner "
>     .. "hoping\n to see the reason for the "
>     .. "strange sounds.\n and you see "
>     .. description(obj2)
> 
> You do this:
> 
> text = join(
>     "You peek carefully around the corner ",
>     "hoping\n to see the reason for the ",
>     "strange sounds.\n and you see ",
>     description(obj2))
> 
> If join was written in C it could do the concatenation
> quickly, and then you'd only be interning the result string. It should
> speed things up considerably. You could even write it in Lua and still
> cut down on the
> number of strings interned using a binary tree approach (I think Roberto
> posted something like that six or eight months ago).

Lua 5.0's table.concat{"Foo", "Bar", "Doh"} is already doing this, if I 
understand correctly what you are saying.
Of course, for static strings like those I give, using .. is more efficient, 
as Luiz answered. This function is more useful when you put programmatically 
the strings in a table, then concatenate them at once, with an optional 
separator.

I was thinking about this issue since the LTN of Roberto, wanting to write 
something about it but always postponing. Seems like a good opportunity to 
give a go :-)

If compared to C, which is unfair (to compare a low level compiled language 
to a high level scripting one), Lua's *core* support of strings is strangely 
lacunar. I don't know for other scripting languages, my experience is too 
limited here.

Basically, you can declare and set a string, use it as index, compare two 
strings (case-sensitive), concatenate them.
That's about all. No base support for getting the length of a string, 
nothing to access an individual character, even less, given the immutable 
nature of strings, the capability to change a given char.
So, unlike C, you can't write a string library in Lua, you have to rely on C 
to build it.

Now, these are limitations we can live with, I understand (more or less) the 
deep nature of string in Lua doesn't allow it, I don't ask to change that...

But perhaps the support of a core "buffer" type would be useful.
I don't dream, this is hightly unlikely it will ever happen, but still, it 
may be useful to have it in the base Lua, allowing semi-transparent 
conversions between this type and the string type, and (perhaps) avoiding 
the overhead of function calls (using eg. stringId[5, 9] to get a sub-
string).
Plus it may allow to avoid duplicating functions for regular strings and 
buffers (libraries check what they get), literal strings used only in buffer 
manipulations would not be hashed/garbage collected (ie. would exists only 
at source/bytecode level), etc.

Now, realistically, this would be a library, the buffer being a userdata.
Such a buffer would be a C-like (or more probably C++-like) string: you 
define it with a given size, but it can grow automatically (or not, if told 
not to).
You can get its size and length, (safely) concatenate strings to it, access 
any char or sub-string inside (nil if not yet set? or initialize at 
predefined value) as read or write (behavior to define when setting a byte 
inside an undefined area), read and write part or whole buffer from/to a 
file, etc.
It would be perfect to build a string (which can be binary) piece by piece 
(eg. creating an image before writing it to a file, or the Life sample).

Perhaps the need for such a feature isn't general, ie. some applications may 
have no use for it. So putting it in the core isn't such a brilliant idea 
(and probably not doable anyway).

But to have such a library would be a good idea. I may give a try someday, 
if nobody does it before, and if nobody destroys the idea as useless :-)
I have first to write detailled specifications, to have an idea of where I 
go.

-- 
--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--=#=--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/