Re: Searching for prior art.

[Flyer31 Test <flyer31@...>]

Hi Francisco,
if you really want to change the VM for this, I cannot help you - this
sounds very difficult from my point of view, much about my level (and
of course strings / storage is really VERY deeply at the base of Lua).

But if I understand you correctly I had a similar problem 2-3 months
ago, and then I programmed my own string bufffer / strbuf lib, which
meanwhile really runs perfectly and very fast and avoiding any string
allocation / garbage collection for the strings handled by this strbuf
lib.

This I did with metafunction support, I described it roughly 1 month
ago here in the forum.

I had the problem that I had the ambition to cut down text files for
programming to my small STM32 controller devices, which have anly
about 64kByte RAM available for Lua. (Lua32 5.3.4). The flash
programming for these devices is a bit "cumbersome" - the text file
has to be cut into pieces of 8bytes, and then these 8 byte blocks can
be programmed "piece by piece", every 8 bytes needs about 100usec
program time, and larger block programming noct possible. So I created
"myriads" of small strings. With Lua string handling, this worked fine
until second invocagtion of garbage collector (and the RAM heap space
always increased dangerously to my 60k heap limit...). After second
invoke of GC LUa then would crash typically. At first I thouhht about
some general error in my Lua program, but if I tried the same code on
a "large system" (Win 10 PC, Lua64bit), then all worked perfectly
fine...  (as far as I could judge, of course this was only "similar
testing", no real flash controller programming on the Win10 PC ...).

So finally I ended up with this strbuf lib. This allocates a string
with up to 255 Bytes (my restriction, but you could also make this
larger of course). somewhere on start with sb.setup command as global
variable.

Then all small string cutting and so und is done with 2-3 such strbuf
buffers... . And my programming lib (also self written of course) can
handle either Lua strings or these strbuf parts...  So now I use only
the strbuf parts for this - this works very perfectly now. My heap
stops at ca. 20-25kB and does NOT grow further, and programmung runs
"in a flash " (but of course anyway qiuite slow compared to other
possible memory RAM things you could think doing with RAM memory and
such small strings... but all rungs now withoug creation of "Lua
string snippets", by strbuf buffer just always overrides until it is
used ... ).

The writing of this I made very similar to the very nice meta function
example in Roberto's book "Programming in Lua" (Bit array meta
example).

I did some VERY minor changes to the VM for this, but these were NOT
necessary changes, just some things to make it even more perfect and
smooth (I also reported this about 1 month ago in message list, please
just check my last posts, I think quite easy to find).

Maybe this info is somehow helpful to you - all the best!

On Mon, Jan 10, 2022 at 11:33 AM Francisco Olarte
<folarte@peoplecall.com> wrote:
>
> On Sun, 9 Jan 2022 at 18:40, Flyer31 Test <flyer31@googlemail.com> wrote:
> > Can you give some more info about the intended application?
>
> More speed, as my code is full os small strings and this could try to
> avoid dynamic memory usage, but as pointed above it is a no go unless
> I do some more fundamental changes.
>
> > And do you want to write this string extension yourself? In Lua or in
> > C? Or are you looking for a string extension lib "ready to use"?
>
> Is not a string extension, is a change in the VM, but as I skipped a
> problem, pointers invalidating on stack reallocations,  I'll have to
> fully reconsider it ( there is other problem with tables but I had
> this thought out ).
>
> Francisco Olarte.