lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


In my opinion if, uring the traversal, you assign a null value to an existent field on the table, this also causes an undefined behavior for iterators: it causes the key to be unassigned, the table may be flushed/cleaned/reindexed.
Any kind of modification of the content of a table, even on its existing keys, can cause this behavior, including in sequence-tables (tables indexed by positive integers in a continuous range, where assigning too many nil values to some positions could as well remove the integer-based array and force the remaining keys to be indexed in the hashed part instead). Setting an index position to nil and back to the original non-nil value may as well not restore a state usable for existing iterators as the table could be reorganized differently.
If you want stable iterators, on a table you intend to modify, you always need to create a shallow copy of its key index and their values (numbers, string, simple types), or references (to tables/thread objects/userdata...). This requires making some allocation. In Lua there's no way to makr a table to be readonly and create instantly a new shallow copy for the modifiable version, so that the readonly version will remaing valid as long as there's an iterator on it. But that's something that could be part of tables implementations and internal implementations of iterators (to automatically create the shallow readonly copies and free them up automatically as soon as all iterators have reached the end or the iterator has become unreachable and is garbage collected. This would require more work in the garbage collector. But tables in Lua are used so much that it would be very interesting to integrate that in the Lua core itself.
This would finally completely remove the "undefined behavior", would suppress various security risks (notably denial of services caused by endless loops, or crashes caused by unbound allocations of memory and the code exhausting all usable resource quotas).
Tables are still the weakest part of Lua, making it very fragile, and easily attackable. This means that Lua is hardly usable for online web services in shared worker threads used by many concurrent users Lua is TOO simple, but TOO easily attackable, and this unsafe status means that many projects have abandoned it for use on the web, or have had to implement stricter resource quotas checks to avoid crashing servers, but in that case lot of pages can fail to render at any time, unpredictably. And it's impossible to enforce any safe environment and stability: Lua is then outlawed for any critical missions, notably in applications using any kind private user data, and in fact this includes almost applications oàn the web, so user data collection is unavoidable and frequently even required by law): Lua is too dangerous except for non-critical apps using *only* public data, and with failover mechanisms if one server crashes.
But this requires lot of redundancy and implies additional costs which can become huge: the possible minor "gain" offered by the simplicity of Lua is completely lost. There are safer alternatives, some of them are wellknown and popular (and also have more developers working with them, or learning them, and companies investing in them): Python, Rust, _javascript_ are evident (but Java  and even C++ can be safer). If Lua does not solve this problem, it will fatally be abandoned by many people.

Le ven. 4 juin 2021 à 18:53, Tom Sutcliffe <tomsci@me.com> a écrit :
Hi list,

A question occurred to me just now (it is Friday afternoon after all) - the Lua manual https://www.lua.org/manual/5.4/manual.html#pdf-next states that

"The behavior of next is undefined if, during the traversal, you assign any value to a non-existent field in the table"

Which I understand the general meaning of, but the manual's wording got me wondering. Normally "undefined behavior" in the context of a programming language means "your code may do absolutely anything including crash or evolve into Skynet". With Lua however I've always understood that (unless you use the debug library, or other native calls into 3rd party libraries) it should never be possible to write Lua source which compiles but crashes the interpreter, or otherwise trashes the native interpreter state. I'm ignoring invalid bytecode here, as that's understood to offer no guarantees. And of course there have been a handful of bugs over the years that let you do that, but those were clearly bugs and got fixed.

As far as I can see, this is the only place in the manual that uses the phrase "undefined behaviour", so there isn't anything else to compare it to.

So, what exactly are the guarantees on what `next` might do if you violate the assignment constraint? "Not correctly let you traverse the entire table" is a given, but what else? Get stuck in a loop hanging the interpreter? Throw an "invalid key to next" error? Actually crash? My guess would be "not correctly traverse", based on my rough understanding of how table traversal works, but I'd be interested to know what the intended behavior is, and of course if anyone happens to know how any particular Lua version actually behaves.

Cheers,

Tom