Lua 5.0 removed the magic, making it smaller and more flexible.
It's good to know the history. I'm not convinced that 'in' is really the best token for the invocation of something as sophisticated as the generic for, and this makes it clear why it was used -- 'in' was already established as a control structure keyword. In the service of making a language that retains the clarity of 4.0 along with the flexibility of 5.0, I'd be tempted to reclaim "in" for table iteration, and add a new token to be used with the generic for. So it would actually be "in" that was used as a trigger for syntax sugar, expanding to something like:
for k,v in t do --> for k,v over pairs(t) do
The scripting language I've been using for my own projects has been steadily evolving away from vanilla Lua, though thus far I've yet to add any breaking changes. But this one does tempt me.
-Sven