lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On 05/01/2015 17:31, Roberto Ierusalimschy wrote:
The numbering is related to different test suites, one for each
different module in my "patchwork" library.

So, each number is a completely independent run of Lua?

No. There is just one interpreter process running for the whole time. It is just a master script calling other scripts (each one containing a test suite for the corresponding module) using "dofile" and collecting the info reported by the individual scripts.

Then, there is an something interesting: The "distribution of slowness"
is far from smooth.  Several tests are extremely slower in 5.3 than in
5.1/5.2 (by orders of magnitude), and most tests run in around the same
time, but there is not much in between (say, 3~7 times slower).  What do
these tests have in common, different from the others?
Moreover, memory is not always related to time. Some of these tests use
much more memory in 5.3 than in 5.1/5.2, but for many, the difference in
time is much higher than the difference in memory. So, maybe the problem
is not directly related to memory.

I think I isolated the problem, after delving into that 20kB single-module unit testing framework of mine. It's most probably my fault, but an interesting interaction of causes.

The ultimate culprit seems to be an inefficient pattern used in a "match" call deep inside an error handler for xpcall.

That pattern is used to parse the stack trace of errors and extracting some info from it.

The problem is that my unit tests were designed for Lua 5.1/5.2. Since some features of Lua 5.1 were removed in 5.3, running the tests in 5.3 caused hard errors inside the deepest routines of the modules and they were caught in that "xpcall". Since the framework wasn't very well tested against those errors, they simply were gobbled up and not always reported to the user. They only added wasted time because the error handler tried to parse their stack traces. Of course under 5.1. there were not such errors (e.g. table.unpack being removed from the basic library) and so no slowdown happened. And since 5.2 was compiled out of the box with compatibility with 5.1 on, even in 5.2 that slowdown didn't happen (!!!).

I'm not 100% sure my analysis is completely accurate (too messy code in that framework - I just cut it down with a machete until I reproduced the problem also in 5.1 and 5.2), but it seems very likely.

I hacked away that framework ~2 years ago (I badly needed a more consistent way to unit-test my modules, which were about to grow wild) and I almost never touched its code since last year (it's a 20kB monolithic, badly documented "monster"). I planned to clean it a bit, but never found enough time. I knew it would haunt me eventually :-(

I need some more time to test why that pattern was so slow (I still have to remember exactly what I used it for! Reverse engineering one's code - that will be fun. A reminder to myself on why one should never leave his code undocumented for too long! :-)

Sorry for the noise!


-- Lorenzo

-- Roberto