› unit testing Syndicate content Syndicate content

» Shell helper: running KDE unit tests (ctests) the easy way

Thu, 03/26/2009 - 03:09

Unit tests are in my eyes a very important part of programming. KDE uses them, KDevelop does - the PHP plugin I help writing does as well. cmake comes with a ctest program which does quite well to give you a quick glance on which test suite you just broke with your new fance feature :)

But I am very dissatisfied with it. Right now I usually do the following

  1. # lets assume I'm in the source directory
  2. cb && ctest
  3. # look for failed test suites
  4. cd $failed_test_suite_path
  5. ./$failed_test_suite.shell | less
  6. # search for FAIL
  7. cs
  8. cd $to_whereever_I_was_before

That’s pretty much for just running a test. Especially all that cding and lessing became very tedious. Tedious is good, because I eventually fix it:

introducing kdetest

I wrote a bash function (with autocompletion!!!) called kdetest. Calling it without any parameter will run all test suites and gives a nice report of failed functions at the end. Here’s an example (run via cs php && kdetest).

  1. kdetest
  2. # ... lots of test output
  3.  
  4.  --- ALL PASSED TESTS ---
  5. ...
  6. PASS   : Php::TestCompletion::implementMethods()
  7. PASS   : Php::TestCompletion::inArray()
  8. PASS   : Php::TestCompletion::cleanupTestCase()
  9.  
  10. 143 passed tests in total
  11.  
  12.  --- ALL FAILED TESTS ---
  13. FAIL!  : Php::TestCompletion::newExtends() Compared values are not the same
  14. FAIL!  : Php::TestCompletion::updateExtends() '! forbiddenIdentifiers.contains(item->declaration()->identifier().toString())' returned FALSE. ()
  15. FAIL!  : Php::TestCompletion::updateExtends() '! forbiddenIdentifiers.contains(item->declaration()->identifier().toString())' returned FALSE. ()
  16. FAIL!  : Php::TestCompletion::updateExtends() Compared values are not the same
  17. FAIL!  : Php::TestCompletion::newImplements() Compared values are not the same
  18. FAIL!  : Php::TestCompletion::updateImplements() Compared values are not the same
  19.  
  20. 6 failed tests in total
usage
  • kdetest, i.e. without any arguments runs all tests in this directory and below
  • kdetest path/to/test.shell ... runs that test suite only, ... can by any argument the test suite accepts.
autocompletion

kdetest comes with full support for autocompletion of tests and functions, for example:

  1. milian@odin:~/projects/kde4/php$ kdetest TABTAB
  2. completion/tests/completiontest.shell     duchain/tests/expressionparsertest.shell  parser/test/lexertest.shell
  3. duchain/tests/duchaintest.shell           duchain/tests/usestest.shell
  4. milian@odin:~/projects/kde4/php$ kdetest duchain/tests/usestest.shell TABTAB
  5. classAndConstWithSameName          classSelf                          interfaceExtendsMultiple           staticMemberFunctionCall
  6. classAndFunctionWithSameName       constAndVariableWithSameName       memberFunctionCall                 staticMemberVariable
  7. classConstant                      constant                           memberFunctionInString             variable
  8. classExtends                       constantInClassMember              memberVariable                     variableTwoDeclarations
  9. classImplements                    functionAndClassWithSameName       memberVarInString                  variableTwoDeclarationsInFunction
  10. classImplementsMultiple            functionCall                       newObject                          varInString
  11. classParent                        interfaceExtends                   objectWithClassName
the code

You can find the code below, or you can obtain the most up-to-date version on github. Just head over to my shell-helpers repo and peek into the bash_setup_kde4_programming file.