Alternative string concat (was "out of memory ")

["Thomas Wrensch" <twrensch@...>]

This is in response to the fragement of the "out of memory" thread that
is discussing the problems with doing string concatenation and possible
alternatives.

Since the speed of the string .. operation is slow (a problem shared by
several languages) you could fix the problem by increasing the speed of
the existing .. operation or you could use some other, faster technique.

The problem with increasing the speed of .. is that it is hard given the
current structure of Lua strings, garbage collection, etc. The problem
with the second is that you have to do it yourself.

My suggestion would be to write a function "join" or some such that does
just what you want: it takes some number of strings as arguments and
returns a string equal to all those strings concatenated together.

Thus, rather than doing this:

text = "You peek carefully around the corner "
    .. "hoping\n to see the reason for the "
    .. "strange sounds.\n and you see "
    .. description(obj2)

You do this:

text = join(
    "You peek carefully around the corner ",
    "hoping\n to see the reason for the ",
    "strange sounds.\n and you see ",
    description(obj2))

If join was written in C it could do the concatenation
quickly, and then you'd only be interning the result string. It should
speed things up considerably. You could even write it in Lua and still
cut down on the
number of strings interned using a binary tree approach (I think Roberto
posted something like that six or eight months ago).

  - Tom


>>> roberto@inf.puc-rio.br 10/07/02 06:23 AM >>>
> What's wrong with my suggestion to use extra allocation and reference
> counting for strings?

Extra allocation: you can only use extra allocation if you allocate
space
for variables, not for values. In Perl, each variable has its own
buffer,
and a string assignment actually copies the string from one buffer to
the other. So it can do extra allocation in a variable's buffer. In Lua,
each variable points to a string; assignment assigns the pointer. So,
it doesn't make sense to have "extra allocation" in the space used by
a fixed string.

The scheme used by Perl makes some string operations much faster than
in Lua, but makes several other operations much slower. For instance,
the simple creation/release of local variables take some time. (If you
simply declare a dummy `my' variable in a function it slowdowns the call
by more than 10%).

Reference counting: Lua is not statically typed. To do reference
counting for strings, you will have the overhead of reference counting
everywhere. Then it is better to simply change the GC mechanism to
reference counting.

-- Roberto