Re: Memory Size of 1 Mio Point Polyline table?

[Rolf Kalbermatter <rolf.kalbermatter@...>]

I did not mean to say that it “will” be a problem, but it definitely can be if the code uses typecasts or other similar C tricks. And the performance degradation from unaligned data access is a real concern on some non-Intel CPUs, and Lua can be used on pretty much any hardware that has a C compiler, so it’s not only an academic concern.

Sent from my iPhone

> On 23 Oct 2022, at 13:39, bil til <biltil52@gmail.com> wrote:
> 
> Not "always", this will depend on the prgramming style, or memory
> attributes like "packed". It is no problem to force the C compiler to
> skip alignment.
> 
> Just of course all variables forwarded directly to C need to be
> aligned. Except strings, which are anyway "byte aligned". But as the
> numbers are all transferred in form of copies, it should be no problem
> to achieve this. And Lua does NOT allow direct access to tables, only
> through lua_getfield / lua_setfield on single table elements, so here
> also no problem should come up... .
> 
> (This would only lead to problems, if e. g. Lua would present C the
> way to address directly the indexed part of a table, but as far as I
> know this is impossible / would impede any security rules of Lua...).
> 
>> Am So., 23. Okt. 2022 um 10:40 Uhr schrieb Rolf Kalbermatter
>> <rolf.kalbermatter@kalbermatter.nl>:
>> 
>> You shouldn’t forget for alignment. If the data is an 8 byte element and always followed by a data type integer it really won’t matter if that data type value is a byte, word, long or quad (int64). Unless you change the default alignment too, the compiler will always align the 8 byte data value on an address that is a multiple of 8 bytes, wasting the unused bytes in between.
>> 
>> And just blindly enabling byte packing usually has some serious implications if the code is not completely written with that in mind and on non-Intel x86 platforms potentially huge performance implications for unaligned access of memory addresses.
>> 
>> Sent from my iPhone
>> 
>>>> On 23 Oct 2022, at 07:25, bil til <biltil52@gmail.com> wrote:
>>> 
>>> Thank you for the PDF version of Roberto's "Lua performance tricks"
>>> article from Lua Gems.
>>> 
>>> After this extremely nicely presented tragic-philosophical start about
>>> modern software programming in the first paragraph, it gives me really
>>> very interesting insights into the storage of Lua. As I especially am
>>> just interested in tables, it is especially this Polyline
>>> construction:
>>> 
>>> polyline = { x = { 10.3, 10.3, 15.0, ...},
>>> y = { 98.5, 18.3, 98.5, ...}
>>> }
>>> 
>>> It is stated that this needs 24MBytes for a 1 Mio-Point Polyline (see
>>> errata, there is a misprint of "kBytes" in the original article).
>>> 
>>> I assume, this depends strongly on the C compiler used?
>>> 
>>> So 24MByte presumably for a C compiler for Lua64 using 'int' fo  32
>>> bits to facilitate to "uplift 32bits software to 64bit world" (as it
>>> does MS Visual Studion C++ e. g. for the creation of Win64 bit
>>> software)?
>>> 
>>> And I assume this was before Lua started with 64bits integers / lua
>>> integers, so possible in Lua5.1 times? (8bytes per double, plus 4
>>> bytes per type, so 12 bytes per Lua number?).
>>> 
>>> And after Lua integers appearing (I think Lua 5.2) this would possibly
>>> even have increased to 32 MByte? (8 bytes per double, plus 8 bytes per
>>> type info, so 16 bytes per Lua number?).
>>> 
>>> And after Lua 5.4, where the type info shrinked to 1 byte, if I
>>> understand this correctly, this shrinked now to 18 MByte (8bytes per
>>> double, plus 1 byte per type info)?
>>> 
>>> And for Lua32 the numbers would be quite identical, except there of
>>> course 4 byte per float, and int also remains with 4 bytes - so
>>> approximately halfed memory usage?
>>> 
>>> Could some "Lua internals specialst" tell me, whether this is correct
>>> understanding from my side?