program selection system

Abstract

Performance testing our applications is either neglected or we use expensive and time consuming to learn commercial tools that often produce so many metrics and raw data that it can be difficult to derive meaningful information to improve application performance. What can agile teach us about doing performance analysis? This talk provides an overview of available open source performance testing tools and discusses minimalistic approaches to testing web sites and web services using tools including JMeter, Grinder, WebTest, SoapUI and a Groovy DSL.

__The Agile movement has largely ignored the experiences of CTOs and senior leaders in an Agile enterprise. In this session, we move toward a pattern language of CTO-level Agile practices and underlying principles, with supporting or constraining forces. Presenters will explain survey data and a CTO panel will contribute their stories.

In this session, three representatives of an agile team will show how an automated build that executes robustness, scalability and performance tests helped them drastically improve the quality of their highly concurrent application server. They will also show how the team configured such builds in their continuous integration environment as well as what performance and robustness metrics they monitored. Finally, the team will show how valuable and effective this investment has been for capturing bugs and performance-related issues very early in their development process.

Intro

A small team, working in an agile group, discusses their experience on how they pushed continuous integration and automation for performance, scalability and robustness builds of their VoIP stack, the pros and cons of the approach taken to automated testing of the so called non-functionals and the benefits (and drawbacks) gained by automating such tests. They also discuss reasons why these tests have been written and automated early in the development process and consequently re-factored and improved over time to meet new customer needs.

Problem statement

Unit testing is often not enough to have confidence that the delivered software complies to the desired quality. More often than not, it is necessary to add multiple layers of testing. Needing to complete the user stories of scalability and performance measurement and improvement, the team built confidence through automated builds that tested robustness, performance and scalability. These automated builds currently run in an environment created using open source tools and custom built software. The team is now able to deliver stories like:

• When taking a VoIP stack from a single node server to multi-node server, repeatedly test the scalability over time.

• Test the robustness of the server product with likely scenario usage;

• Measure performance throughput over time using various persistence models.

• Automate such tests and continuously monitor the output as a measure of the quality of the product over time.

Attending: who and why

Those interested in pushing their continuous integration environment further and automating as much as possible will benefit from the experience report of how this team achieved automated and continuous robustness and performance builds. Those with little to no experience of continuous builds may see the art of the possible and see how automating these processes allowed the team more time to complete their customer stories while continuing to monitor metrics in the background and to record performance history over time. It also allowed the team to discover early and often about bottlenecks and leaks that had been introduced and the effect of bug fixes on performance.

At the end

Q&A will follow at the end of the presentation to conclude the session.

At a Glance
This session provides ideas and approaches on how to promote a culture of continuous improvement and perpetual learning among teams of software developers in a corporate setting. It provides specific examples of approaches that have been successful at large companies such as Travelers Insurance and Google, focusing not only on how to spread knowledge, but also on how to evaluate progress and measure success. It also presents some lessons learned, and looks to the audience to share their thoughts and experience on where to go from here.