Exposing the string buffer used internally by `string.gsub` and `table.concat`

[Lars Müller <appgurulars@...>]

Both string.gsub and table.concat are internally efficiently implemented
using an exponentially grown string buffer. To efficiently "transform" a
string, you can usually use gsub; to concatenate strings, you can build
a table and concat it. Unfortunately the string buffer remains
unexposed, so you always have to use either function. gsub is often
unsuitable - in particular when you are "generating" a string rather
than "transforming" one, or if your "transformations" are too
sophisticated for gsub. In these cases, you will usually have to resort
to concat, which forces you to build a table. If you are building
strings character-wise, this will use 8 - 4 times the memory of the
internal string buffer on a 64-bit system. Additionally, it requires you
to build a table, even if everything could be streamed directly into the
buffer.

Thus I suggest a new string library function called
string.from_iterator(...) or the like; it would take as parameter(s) a
for-loop iterator func, state, initvar, closevar and would return a
string, internally using a buffer to build it.

This could e.g. be used to conveniently build a string using
string.gmatch: string.from_iterator(("abba aba aa bb"):gmatch"ab*a")

- Lars