LuaUnit/BTD future

[Wim Langers <wim.langers@...>]

As the 'Lunaticos' who follow what's happening on LuaForge probably already know, I've taken the initiative to fork (a presumably dead) LuaUnit to bring it on to par with Lua itself and to further enhance it. The fork was necessary because I lost contact with the original developer. Despite all this I wanted a compatible solution to make the transition (for interested users) as painless as possible.
I think that, unless somebody tells me otherwise :o), I succeeded in that goal.

For me the unit testing framework missed still following features :
- don't abort framework under any circumstance, but catch/respond to all errors in a controlled way
- provide maximum isolation between test methods and framework
- ability to kill a test if it doesn't terminate in time
- run tests in parallel to optimise tests which are waiting for some event/condition/time

Lua Lanes provides most of these features and should help in the others. But just as I finished integrating it, and being quite happy with the result, I find that a number of LuaUnit test cases don't work anymore...

So this is basically a cry for help (or a plea for acceptance of my solution :o), see bottom of message ) to the users of LuaUnit/BTD.


Description of the problematic of running a test (e.g. test:run(parameters)) :
1. test runner possibilities :
- TestClass and/or TestClass:testMethod are already present in _G
  - test:run()
    >> will pick up and run all these tests
  - test:run('TestClass',...) or test:run('TestClass:testMethod')
    >> respective tests will be run
  remark : if the tests rely on helper functions present in _G, these will also work properly
- XClass and/or XClass:xMethod are already present in _G
  - test:run()
    >> will not run any tests
  - test:run('XClass')
    >> will run all XClass:testY methods
  - test:run('XClass:xMethod)
    >> will run the specified test
  remark : if the tests rely on helper functions present in _G, these will also work properly
- same as the above but tests are not in _G but loaded through require (which is also an original LuaUnit option)
  remark : if the tests rely on helper functions present in _G, these will NOT work properly

remark : already present in _G may be a result of :
- running other scripts
- the actual test script combines test classes and the test:run method (e.g. use_luaunit.lua script)

2. problems :
- up until now all this worked as it does under LuaUnit
- now comes Lua Lanes, and the plot thickens...
  - need Lanes to provide robustness required of a testing framework
  - but lose the global environment of the calling script (e.g. use_luaunit.lua)
  - could copy _G to all Lanes (might be hundreds because ALL test methods might run in parallel), but
    - obviously shouldn't overwrite existing functions/data in Lane
    - don't know which functions/data to copy (there might be helper functions and/or data)
    - debug.traceback won't give relevant (to actual failing test method) information anymore ?
  - could fall back to just requiring the calling script in all Lanes (e.g. use_luaunit.lua),
    but this doesn't take into account extra test cases or helper methods which might be in _G

remark : the problems, obviously, are not related to Lua Lanes but to the concept of multi threading and _G

3. solution :
- drop the test:run() option as it is now
- (I was thinking of having test:run() recursively scan directories to pick up all test classes...)
- all test classes and/or test methods should be found through require
- changes to existing test classes are minor (if at all necessary)
- best practices = don't fiddle with/rely on the global environment ?
- best practices = organise test classes in appropriate files ?

remark : btd.lua.test, through the use of require, supports paths (e.g. 'dir.dir.TestClass')

4. question :
does anyone even use the (now) problematic functionality ? does anyone mind ? have alternative ideas ? solutions ?


Wim Langers