Very large string loading into Lua

[Thomas Harning <harningt@...>]

I was doing some benchmarking on a base64-encoding routine and was  
hoping to change it from the form:
  ptr = malloc(large-enough)
  for all data
     base64-decode

into
  init luaL_buffer
  for all data
      if not enough buffer left, luaL_addsize and get new ptr
      base64-decode

It turns out that this isn't so nice, particularly when the data size  
is much larger than LUAL_BUFFERSIZE  (on order of 10s of MB)

Has there been any thought on how one could create an arbitrary string- 
region that one could write to and then finalize?
Perhaps even be able to convert a userdata object into a string in  
place (and freeing it if the string value already existed due to  
interning)

Another feature that would probably be useful (particularly for string- 
building) is for userdata to be resizeable...

Any thoughts?