How to be Agile with Fragile Large Legacy Applications?
, Ainsley Nies
Can you imagine applying agility to a huge legacy code base over a decade old, supported by 100+ geographically distributed developers, working on thousands of classes and millions of lines of code?
In this talk we share the stories of experience with three clients – the expected, the unexpected and what was learned, and how we enabled them to go from fragile to agile. We will cover our approach and the key practices we discovered to be invaluable for success with large Legacy applications.
Can you even think of applying agility to legacy code that’s over a decade old and supported by team with hundreds of developers working in distributed different geographies on thousands of classes and millions of lines of code?
In this talk, we share real-world experiences and learnings from how several teams were enabled to go from fragile to agile. These teams were working on large-scale code base varying from 3000 classes to 6000 classes and close to a million lines of code - some of it almost a decade old! In each case, developers were making changes that rippled through the class structure, with no tests in place - a high-wire act with no safety net resulting in regressions and lots of unit level bugs found in production.
Each of the customers had typical legacy challenges of reduced productivity, maintenance nightmare of legacy application, increased number of production defects, lots of regressions found, increased QA team and cycle, loss of customer satisfaction. The mission of each engagement was to address legacy challenges without a major disruption to the business.
We will describe the hurdles encountered in implementing agility in complex, large-scale legacy projects. Some of the problems encountered were expected, but some caught us completely by surprise! We will cover how we addressed these problems and surprises, the approach we followed, the tools and Key Practices. We will also share how the Key Practices guaranteed the sustainability of changes in the organization/culture without overloading the team or organization.
Some of the metrics that we gathered before and after implementation: - QA cycle time reduced from 7x to 3x, where x is the development time. - Number of unit-level defects found in QA was reduced by 20% - No more integration hells - due to rapid feedback with continuous builds - Increased productivity by at least 30% (by breaking the big monolithic project into 16 smaller projects) - Reduced the application complexity by 10-15%
We will be covering our Key Practices: 0. Inspect and adapt 1. Go slow To Go Fast 2. Big is Bad - break into smaller manageable projects 3. Establish safety net 4. Provide Extreme feedback 5. Adopt incrementally 6. Reduce complexity 7. Sustain the process with “tiger-team”
Each of the above practices is shared in the format of anti-pattern with examples, anecdotal evidence and recommended solutions. References used in developing these practices will be included. (e.g. Michael Feathers, Linda Rising etc..)
Only working software becomes Legacy code. Long lived code is hard to change and is compared to “Antipattern - Lava Flow”. There are practices that can reverse-engineer the software aging and allow even extremely large legacy code bases to become easier to change. More Details on some of the Key Practices:
Big is Bad: Huge monolithic projects are bad in many different aspects. They cause productivity loss, and longer time before enough feedback. Even if we establish continuous integration on Large Legacy code base, the time to run the tests and give feedback may not be fast enough. This by definition defeats the purpose of Continuous Integration. We have helped few customers solve this challenging issue of longer feedbacks and also help localize the errors, by dividing one monolithic project with 5000+ classes into 15 smaller projects. Having said it is not always easy to break monolity legacy projects with dependency graphs, since these legacy code may be full of circular dependencies which are hard to break. Our customer situation was no different, they had lots of circular dependencies as well. Previous attempts to break this projects have been failed and have estimated time of 2-3 months to achieve the goal. Despite the circular dependency and hard to break the project into smaller projects, we had achieved our goal in approximattely 2 weeks. We used symbolic links technique on the linux machine of CVS (Version control system) to create smaller projects - despite the issue of circular dependencies.
This one step itself, dramatically improved the performance of the Continuous Integration builds and enabled to quick feedbaack many times throughout the day. The team had productivity gain of 15% atleast.
Go Slow To Go Fast: The Legacy Dilema as stated nicely by Michael Feather in his book “Working Effectively with Legacy Code” is “When we change code, we should have tests in place. To put tests in place, we often have to change code.” To meet this Catch-22 issue of Legacy applications, one always feel the need of Safety net around the Legacy code. Safety net could be created with various approaches. Starting with this assumption, we applied the practice of creating Safety net on Large Legacy code base for a customer and setup continuous integration in place. Our results caught us completely by surprise, and were not what we expected. Team who didn’t have any process of Continuous Integration and or responsibility of tests, were overloaded. The team was overloaded with content, process and feedback, which they were not prepared for. As the team suddenly has to own much more tests, Continuous Integration builds taking too long, scripts throwing out of memory errors, etc..etc.. So our conclusion was this practice which would easily work on small legacy applications, does not scale well with large Legacy applications. and Legacy Applications are different by nature inherently and same guidelines/assumptions cannot work. Our conclusion was to establish the process first with the team, with only small safety net on large Legacy applications. Once the team have mastered the process, and are mature to handle more, or perhaps after breaking the Big Monolithic project into smaller projects, add more safety net tests on Legacy Code. e.g. I like to compare with is - Do we just decide to run a marathon and start running it? No, first we have to start running 1 mile, and train our body towards running full 26.2 miles. Adopting practices to be Agile with Fragile Large Legacy code is like a Marathon and not a Sprint.
We would like this session to be 45 mins.
We will begin by reviewing the issues and challenges with Large Legacy applications, then share actual customer situations before and after implementation.
In next 30 minutes, we will present the Key Practices and answer questions about them.
We will conclude the session by discussing of how these practices have broad application and are not tied to any particular tool, language, type of application or industry, and will answer any remaining questions from the audience.
Compelling Reasons for this talk: - Large Legacy applications are inevitable and reality; the challenges they impose are difficult to overcome. We consistently see the need for additional ways to address these challenges. - The Key Practices have demonstrated effectiveness in real life situations and present a good example of how to effectively leverage existing knowledge. - All customers using the key practices continue to inspect and adapt to their advantage.
Who We Are - Vandana Shah is the consultant who led the customer engagement work, collected the data, established the primary themes and initiated this proposal. Ainsley Nies collaborated in evaluating the resulting data and formulating the learnings.