Thanks Luiz, that's exactly what I was looking for. A quick comparison, hopefully this is not too far off the mark:
A little correction: LuaMacro 1 was indeed based off lhf's tokenf, but LuaMacro 2 used an LPeg Lua lexer by Peter Odding. Which was a good decision at the time, when it was clear that token filters were unlikely to ever be part of stock Lua. Due to the outrageously good performance of LPeg, it is in fact about as fast as the direct patch to the lexer.