JTestMe - improving test feedback and reducing build times with dynamically defined optimised smoke tests
JTestMe dynamically defines optimised smoke test suites for Java applications - dramatically improving the continuous integration cycle time. It is an Open Source project by ThoughtWorks, hosted on the Codehaus.
Using an embarrassingly small amount of AspectJ, your test suites are instrumented to record which application code is executed during the test (similar to code coverage). When a test runs, any application classes which are executed are recorded into a repository. Super-classes and classes that are not directly called by tests are also included.
When you change any application code, the repository is inspected to determine which tests should be run. These tests can then be run whenever you save a file, as an automatic pre-commit build test suite, or as an automatic smoke test suite for a Continuous Integration tool like CruiseControl.
The benefit is that rather than running the whole test suite, or manually pre-defined test suites, only the tests that need to be run to exercise the changed code are the ones that are run, dramatically reducing the feedback cycle for most applications.
Once the smoke test suite has passed, the full unit, integration, and functional test suites would be executed as is current practice, in order to fully assure the behaviour of the application (as some changes are beyond the view of the executing code).
- Set the stage: some real-world examples of the long pre-commit test issue and long build times [5 mins]
- Overview of JTestMe: basic approach and how we use AspectJ [5 mins]
- Demonstration: taking a code base that has a 10 minute+ “fast” build time, apply JTestMe to reduce to a couple of seconds for pre-commit and smoke test builds (including unit, integration, and functional tests) so the impact of changes is seen “immediately” by developers [20 mins]
- Q&A [throughout]
- Beers and karaoke [6 hours]