program selection system

In this session, three representatives of an agile team will show how an automated build that executes robustness, scalability and performance tests helped them drastically improve the quality of their highly concurrent application server. They will also show how the team configured such builds in their continuous integration environment as well as what performance and robustness metrics they monitored. Finally, the team will show how valuable and effective this investment has been for capturing bugs and performance-related issues very early in their development process.

Intro

A small team, working in an agile group, discusses their experience on how they pushed continuous integration and automation for performance, scalability and robustness builds of their VoIP stack, the pros and cons of the approach taken to automated testing of the so called non-functionals and the benefits (and drawbacks) gained by automating such tests. They also discuss reasons why these tests have been written and automated early in the development process and consequently re-factored and improved over time to meet new customer needs.

Problem statement

Unit testing is often not enough to have confidence that the delivered software complies to the desired quality. More often than not, it is necessary to add multiple layers of testing. Needing to complete the user stories of scalability and performance measurement and improvement, the team built confidence through automated builds that tested robustness, performance and scalability. These automated builds currently run in an environment created using open source tools and custom built software. The team is now able to deliver stories like:

• When taking a VoIP stack from a single node server to multi-node server, repeatedly test the scalability over time.

• Test the robustness of the server product with likely scenario usage;

• Measure performance throughput over time using various persistence models.

• Automate such tests and continuously monitor the output as a measure of the quality of the product over time.

Attending: who and why

Those interested in pushing their continuous integration environment further and automating as much as possible will benefit from the experience report of how this team achieved automated and continuous robustness and performance builds. Those with little to no experience of continuous builds may see the art of the possible and see how automating these processes allowed the team more time to complete their customer stories while continuing to monitor metrics in the background and to record performance history over time. It also allowed the team to discover early and often about bottlenecks and leaks that had been introduced and the effect of bug fixes on performance.

At the end

Q&A will follow at the end of the presentation to conclude the session.