Scala, choosing a test lib
I note here, (partial) results for my testing tool research/evaluation.
Features :
- required
- run for maven
- run from Eclipse via shortcut ( Alt+Shift+X [ S | T | N | any ] )
- I would like
- support single before/after for set of test (run once, eg : start/stop backend connection)
- ability to use ScalaCheck to generate a set of test
- nice assertion syntax (support scala type)
- support for skip/pending test
- to have less dependencies (jar) ass possible
I create a sample project to try test/bdd lib : prj-scala-only
To run test under eclipse, I follow the rules :
- to be able to run test the project have to compile (fully), required
- name the file same as test class (and package sould match directory layout)
To run test under maven : mvn test :
- end filename with Test (or Suite => require custom configuration)
- If testng jar is in the classpath, then surefire (the maven plugin that run test) only run testng file, so in the final project I comment TestNG dependencies and test using it
JUnit 3/4 :
Maven :- Eclipse : via JUnit Runner Plugin
before/after : only on static method, not scala firendly (need to declare an object)
scalacheck :- assert syntax : via mix of SpecsMatchers or org.scalatest.Assertions, or ShouldMatchersForJUnit,…
- skip support : not found how to
- my sample code :
TestNG :
- Maven :
- Eclipse : via TestNG Runner Plugin
- before/after :
- scalacheck :
- assert syntax : idem JUnit
- skip support : via beforeXxx, or annotation, or test dependency chain
- my sample code :
Specs :
- Maven : extends SpecificationWithJUnit, or mix with JUnit (see doc, or sample linked below)
- Eclipse : I can’t run Specs via JUnit Runner Plugin, or as scala application if I follow Specs’ documentation, But works if I use @RunWith
- before/after :
- scalacheck :
- assert syntax :
- skip support :
- my sample code :
- documentation of the site as grow with version, may be, a little clean up should be done.
- Maven : extends JUnitSuite, or use @RunWith, or extends TestNGSuite
- Eclipse : via JUnit Runner Plugin
- before/after : via trait BeforeAndAfterAll/FixtureSuite, but not as friendly (IMO) as TestNG (failure in before => skipped test and not an error that stop every test)
- scalacheck : detect pass/fail with JUnitSuite
- assert syntax : several syntaxes
- skip support : a pending are converted into failure for JUnitSuite and TestNGSuite, but are ignored in FunSuite
- my sample code :
- The documentation is in the scaladoc
I stop here, my evaluation. But If you’re interested to continue, there is other solution like Specsy, ScalaTestCase, assert from Scalaz,…
Conclusion :
I started this comparison because I was not able to use Specs under Eclipse, and to eval alternative. But at the end of the eval I found how to run Specs under eclipse. And I don’t like a lot the too many syntax of ScalaTest (Error when before failed). So currently, I’ll continue to use Specs.
I hope my feedback could help you.
