On Thu, Oct 10, 2019 at 4:21 PM Roberto Ierusalimschy wrote:
> What api would you prefer for next? And how did its current api spoil generic for? Thanks.
Probably 'next' should return first some opaque "index" object, to be used
in its next call, and then the key and the value. (The index object could
be, for instance, an integer with the internal current position of the
traversal, but that should not be in the specification.)
Similarly, the control variable in the 'for' loop should be hidden like
the state, and not the first variable declared in the loop.
index_obj, key, value = next(t, prev_index_obj)
Which problems such index object could solve?
1) As for now, we could not use nil as a table key.
Obviously, new version of next() could make t[nil] valid because next() will correctly handle a nil key.
But do we really need this?
An opaque index object has more advantages than just that. It would also be more efficient, and it would make it harder to make mistakes, and it would allow the contract of the function to be more clearly specified. (What should happen if you call next() with a key that isn't in the table? How should next() behave if you pass it a table key that ISN'T the next one?) Lots of other languages do it this way -- but there, they call that object an "iterator".
2) As for now, we are not allowed to create new table keys while traversing the table with pairs()
Could introducing of "index object" solve this problem?
We do really need this.
The problem is that adding a new key to a table could result in one of three outcomes:
(1) It gets added into the table before the current iterator, so you wouldn't see it show up in the loop.
(2) It gets added into the table after the current iterator, so you WILL encounter it later in the same loop.
(3) It triggers a table rehash, causing the entire iteration history to be invalidated and there's no way to tell what has or hasn't already been visited.
This problem could be solved by capturing the entire list of keys to be iterated at the start of the loop, but that would be pointlessly expensive for the vast majority of loops -- you'd be iterating over the table twice and duplicating all of the keys, which you would then have to GC later. In the end it's simply better to put the burden on the developer to handle the unusual cases, because if you KNOW you need to add new things to the table during the loop it's more efficient to create a second table holding the new elements and then merge it into the original table after the loop is over.
/s/ Adam
