Agile vs. Traditional Software Development: Why is the debate still going on? - Part I: The dangers of bounded rationality
My understanding is that, in practice, successful agilists tend to bring together a number of activities, tasks, and deliverables that are from beyond the boundaries of what may be called “pure agile.” This mixing and matching of software process elements from agile and non-agile (more formal) approaches is a much more practical way of using these methods.
3.0.0
(Bhuvan Unhelkar, Senior Consultant, Cutter Consortium -- in Agile Product & Project Management, Vol. 11, No. 1 -- 2010, Cutter Consortium).
This article is the first of a series that I’ll be authoring for Modern Analyst. Here, and in future articles, I’ll discuss how hybrid, customized models that don’t follow any “pure” method or approach, agile or traditional, are allowing companies to develop healthier projects, and get slipping projects back on track.
My first topic is Agile’s excessive reliance on user feedback to identify and resolve the requirements uncertainties early in the development process. In this article I discuss how, in agile projects, too much significance attached to user feedback may create significant risks of schedule and cost overruns, and how the adoption of traditional analysis practices may mitigate such risks.
Agilists are constantly talking about how Agile practices support getting things right (or right enough) early on via user testing and feedback. In theory, this makes perfect sense: continually incorporate user feedback so you can quickly discard the wrong solutions and find the right ones. In practice, what I’ve learned is that while early feedback is definitely valuable, it doesn’t necessarily prevent project inefficiencies and risks, and may very well create them, when used to replace sufficient upfront analysis.
I’ll use two projects with different profiles to illustrate my point.
Imagine that Basecamp has not been invented yet, and you are trying to create a Web-based project management application focused on making collaboration and communication easy. Your users know what they don’t like in Microsoft Project, and the team is excited to work together to produce a simpler and better alternative for projects that do not require such a complex task tracking tool.
For this type of problem/opportunity, just-in-time analysis, short iterations, and early feedback will typically lead to good results. It’s not difficult to define and evaluate the functionality that is being produced, such as create a to-do list, assign to-do items to team members, etc. Progress is fast, missing requirements or inadequate design solutions are quickly identified, and what didn’t work can be fixed in no time.
Now, let’s take a look at a different real case scenario. Consider a legacy system that supports a complex business process, and took years to be developed, expanded, and enhanced. The system has multiple integrations with applications that either provide data for the system, or consume information it produces. The new system will have to not only satisfy unmet business and user needs (making the application easier to maintain, simplifying tasks, and providing more user-friendly interfaces), but also entirely replace existing functionality and integration points before it can go live.
As users describe and prioritize what they want (new calculated fields that will reduce the amount of work and data entry errors, user-friendly menus for navigation, etc.), the project starts on a positive note, with working software being delivered every 3 weeks, and feedback being used to make the system better.
Fast forward a few months. The system is not in production yet; due to the size of the legacy application that is being replaced, go-live is still a few months ahead. By then, users performing acceptance tests start noticing certain things that aren’t quite right with features they had previously approved. For example, the user story “as a manager, I want to know which tasks have high priority so I can assign them to the next available resource” seemed fine, until a subsequently implemented feature allowed a user to discover that the calculation used to rank tasks by priority does not work in 30% of the cases. “Not a problem”, the developers say, “scratch the priority calculation! Let’s go back to showing tasks by chronological order and letting the manager manually choose which tasks should be given higher priority, as the legacy system does.”
All would be good if it wasn’t for the fact that this is not the only case of a previously requested and accepted feature all of a sudden proving to be "unfit for purpose" after other parts of the system were made available for testing.
More and more problems start to surface, with user stories previously considered “done” having to go through rework not only once, but multiple times, due to new insight caused by added pieces of functionality interacting with each other. While in the beginning the entire team is not concerned, (“change is good, we embrace change!”), soon the volume of modifications required in code is such that the project is now running the risk of failure due to the impact in schedule and budget.
The most worrisome aspect of this scenario is that the changes requested by users reflected needs that may have been unforeseen, but were not unforeseeable. Changes had nothing to do with the “uncertainty inherent in the world”. Rather than triggered by the need to respond to new market conditions, shifts in business strategies, or new insights about user behavior, such changes could be easily avoided by proper elicitation and analysis of existing requirements. The need to modify previously implemented functionality caused waste and rework that could have been avoided if the requirements had been more clearly understood before the programmers started to code the features [1] .
To explain that change is inevitable, Agile practitioners like to quote Humphrey’s Requirements Uncertainty Principle, which states that for a new software system, the requirements will not be completely known until after the users have used it. The untold part of the story is, that in many complex software projects, unless you put sufficient effort into requirements discovery and analysis, requirements may not be completely known even after users have tested and approved pieces of functionality delivered sequentially.
Why is it so? The main reason the combination of “just-in-time requirements” and early user feedback often fails to uncover missing/incorrect requirements is that system thinking is not natural for most of us. Modern physics teach us that the behavior of systems may not be a direct function of the behavior of the parts of a system, but when users are asked to explain what they mean [2] , or are invited to test a feature, they focus on that particular aspect of the solution, without paying enough attention to the nature of relationships with other parts of the system and to non-linearities that may produce unexpected results.
User input and feedback, in special in early stages of a project, may be seriously affected by bounded rationality--the notion that individual decisions are rational within the bounds of the information available, but such information is not perfect, especially about more distant parts of the system [3]. While first testing the task-prioritizing feature, based on the part of the system they could see and know, users made a reasonable decision that the functionality was fit for purpose. Only after their bounded rationality was enlarged (in the presence of better, more complete information provided by the addition of other features) the conclusion changed.
Agile proponents argue that because Agile methods support changing direction, the cost of being “wrong” is much smaller than in traditional, less flexible methods of software development. This conclusion is only true when you are capable of identifying that you are in the wrong direction early on, so the lessons can be incorporated in a timely manner. Unfortunately, that is often not the case in more complex projects.
Agile's philosophy that analysis can be done incrementally, with work broken down into small, achievable "chunks" of functionality, has proved to be wasteful in many of the agile projects I've joined. While implementing these chunks within short periods of time has been beneficial, allowing users to interact with the solution to verify whether it meets their needs, incremental analysis caused the stakeholders and solution teams to ignore the fact that a system is different than a heap or a collection. To truly understand a solution, one must understand the mutual interactions of the parts of the system, whose characteristics cannot be found as characteristics of any of its individual parts.
In a previous article, I wrote:
"As with plan-driven organizations, mature agile organizations are always looking for ways to improve the performance of their software process. Optimization of the development process may require tailoring agile methods to the company’s needs and capabilities, and adding a talented BA to the team is becoming an obvious answer for many of the struggles that sometimes agile teams face, from competing interests among various stakeholders, to lack of a clear understanding of the problem to be solved with the software solution."
Skilled business analysts are trained to think systemically, analyzing requirements in relationship to other requirements, and seeing each piece of a system as a meaningful component of a working whole. While many agile teams prefer to document the solution after it has been delivered, or perform “just enough” requirements analysis for the highest-priority requirements taken from the project backlog, these strategies not always yield the best results. While it’s true that customers and users do not always know what they want at the outset of a project, and that we must be open to change during project execution, that doesn’t mean that just-in-time analysis is always the best approach to understand the domain and identify what needs to be built.
“Model just for now, you can always come back later” may be a good method to explore new ideas and generate innovative products. However, it is well-known that the later in the development cycle major errors are discovered, the more expensive it is to fix the defects. Being open to change should never be an excuse for lack of systems thinking, poor analysis, and inadequate and ever-evolving requirements. While adopting some of the Agile practices may help increase the value of software projects, the use of traditional business analysis techniques may turn out to be a huge step toward keeping a project on track and avoiding costly, high-risk requirements churn.
Read on: Agile vs. Traditional... - Part II: Methods are a means to the end of project success, not the end themselves
Author : Adriana Beal received her B.S. in electronic engineering and MBA in strategic management of information systems from two of the most prestigious graduate schools in Brazil. For the past 10 years she has been offering consulting assistance throughout the software development life cycle for organizations implementing large, complex software projects. Adriana recently joined e-commerce team at Hewlett-Packard. She is also the founder of Projeto 100%, a movement making significant changes in the lives of families trapped in poverty in Brazil.
[1] It’s interesting to see that even though the new system was replacing a legacy application (and consequently inheriting many requirements from the old system, instead of having completely new requirements), it was still hard for users to keep in mind the interactions that would happen between different parts of the system when all the pieces were put together. There was no significant difference between the requirements churn experienced in this project and in other agile projects I joined that weren’t replacing legacy functionality.
[2] "To discover the details behind the requirement, the developer (or developers on project teams which take a pair programming approach), ask their stakeholder(s) to explain what they mean." Source: http://www.agilemodeling.com/essays/agileRequirements.htm
[3] For more on bounded rationality, read ,Thinking in Systems: A Primer.