Workshop on build/test grids and selective testing tools
Agile developers love — and some actually need — to be in a state of flow. For them fast feedback from unit tests and continuous integration tools is not an option. In practice, fast unit tests often lack in coverage and many valuable tests run slow. This may be due to legacy code or the nature of the tests or just the sheer size of the application. Whatever the reason, getting feedback many hours after they submitted the code and the mental model has been blurred is a major impediment to developer’s productivity. To mitigate these issues a variety of strategies and tools have emerged. They primarily fall into three areas:
- Distributed and incremental compilation and code generation grids.
- Distributed test execution grids.
- Selective testing tools that can dynamically construct an appropriate smoke test suite.
The goal of the workshop is to inventory current techniques and tools and to actively imagine promising future directions and features.
An example of this approach may look like the following: We have a cluster of 10 test servers and we assume that an incremental compilation, if any, takes only few minutes. In a typical continuous integration cycle 100 lines of code are modified. From code coverage information we know what tests are related to those lines of code. We also know the historical runtime of each test. We choose 200 minutes worth of most relevant tests and get an overall result in roughly 20 minutes.
Furthermore, we can merge the code and run the tests before they are committed to the repository, thus greatly reducing the risk of a broken build.
Workshop wiki: http://sites.google.com/a/itarra.com/test-grids/
(10 min) Introductions.
(40 min) Mini experience reports (max time per speaker is 5 min). The facilitator will choose the speakers with the intention to cover as much of the space as possible. Folks who have experience using these techniques and tools will share their stories. If you are a tools vendor, please prepare an experience report from an actual customer implementation.
(40 min) We conduct a round-table type discussion focusing on the challenges to widespread adoption of this approach focusing on issues relevant to both implementers and tool developers.
Whenever possible we will capture important data on the easel and later transcribe it to the wiki.
Materials: One easel and large format self-adhesive paper.