Agile vs. Traditional Software Development: Why is the debate still going on? - Part III: Winning Through Integrative Thinking
In the previous article on this series, I described how a project grid can be used to better understand the characteristics of a project during its initiation phase. It may seem like a huge effort to classify each project using that type of grid, but once an organization has established a good classification method that works for its particular context, identifying the project profile becomes a quick process, worth of the investment in time and effort.
Understanding the profile of your next software project is the first step toward making informed decisions regarding the best approach for the development effort. While a lightweight framework such as Crystal or Scrum may be perfect for some types of project (e.g., the flagship Web application for a startup), it’s unlikely that any of the commercial agile models will address all the issues that many software projects face (e.g., larger and distributed teams, reuse of extensive amounts of legacy software, multiple stakeholders with conflicting interests, etc.). Instead of taking for granted that either you find a flavor of agile that will fit the needs of your organization, or you must completely dismiss the use of agile methods, a much more valuable approach is to determine, for each individual project, which agile concepts should be embraced or not. The decision to use an agile practice should come from its ability to improve customer satisfaction, costs, quality, cycle time, and/or productivity in your current project, not from the fact that it’s part of a well-established agile methodology.
As I wrote in part II of this series:
Many agilists argue that you can't pick and choose which agile practices to use and expect an agile method to work, insisting that you must follow all the principles behind the Agile Manifesto, or all tenets of XP or Scrum or Crystal, to call yourself "agile". Their concern is understandable, considering the fact that many companies declare that they are "going agile" while refusing to abandon their plan-driven, budget-scope-schedule approaches (obviously with disappointing results). However, it's unreasonable to expect that the adoption of agile practices will automatically cause a project to generate business value. Agile practices are not intrinsically “value-adding”: they must be aligned to business needs in order to provide true value.
Most arguments against hybrid models are strongly influenced by a belief that the two opposing models, waterfall, and agile, are the two competing realities. The comment below, from a reader disagreeing with an article in which the author defends the hybridization of agile and traditional methods, illustrates this view:
If I understand your thesis, you're effectively arguing for holding on to a portion of the status quo of project delivery: If it's too hard to do completely with agile practices, there's no shame - and in fact, even honor - in retreating to the warm security of planning everything up front, moving work products through successive phases and dining a la carte from the agile buffet.
While there's certainly nothing stopping you from doing so, I have to ask "What's the motivation?" Why not be totally pragmatic and skip agile completely and just go all-in on waterfall? It's easier, right? Promising everything up front, hoping that things go 100% according to plan and you've padded the estimates enough to cover overages? Not bothering to surface problems or issues daily - skipping out on reviews. It's all a bit much, isn't it?
This type of stance is typical of a conventional thinker, one that believes that if your problem doesn’t fit all elements of a model, then you must choose the opposing model. In his book The Opposable Mind: Winning Through Integrative Thinking (from which I borrowed the title of this article), award winning author Roger Martin explains the difference between individuals who are “contented model defenders” and “optimistic model seekers”.
Contented model defenders, explains Martin, adopt a theory and then seek to support and defend it. As contented model defenders accumulate data in support of the theory they’ve adopted, they become more certain that their theory represents the truth, and more content that they have achieved their ultimate goal, certainty. Optimistic model seekers, on the other hand, don’t believe there is a right answer, just the best answer available now. They understand that all models are fallible, and the best current model will be eclipsed in due course, as will its successor models.
When we adopt the perspective of an optimistic model seeker, we stop thinking in terms of choosing between “agile” and “waterfall”, or any other model, for that matter. There is no reason to dismiss a model because it doesn't entirely fit your reality. Why not look at models together, rather than separately, in search for useful answers? Why not borrow from different methods and frameworks, to find a creative resolution to the tension between the two extremes of agile and waterfall?
Maybe your project is a complex legacy system, and because you need all the existing capabilities implemented in the new system before it can go live, the concepts of early delivery of working software and just-in-time requirements won’t add value to this project. Maybe your new system must be well documented in business jargon or it cannot go live due to the severity of the penalties the company would be facing in case of non-compliance, and thus the project needs to value good documentation as much as working software. Should you just ignore other agile practices that could be useful for your project only because it doesn’t entirely fit an agile model?
Just because a project cannot benefit from specific agile practices, it doesn’t mean that other practices found in agile methods, such as eliminating waste caused by partially done work, or getting user feedback for implemented features as early as possible, must be discarted. The concept of integrative thinking—the ability to hold two opposing ideas in our mind at once, and then reach a synthesis that contains elements of both but actually improves on each—has a good example in this TED talk. Graham Hill describes how he looked at two existing models, being a meat eater (which causes environmental and health problems), and a vegetarian (which would take away the pleasure he feels eating hamburgers), and instead of being contented with the opposing options that he was being pitched, found a pragmatic solution: become a “weekday veg”, which allowed him to cut 70% of his meat intake while still being able to enjoy a nice steak from time to time.
Approaches for software development should not be looked at as an "either-or" proposition, but rather as a "both-and" situation. Granted, it's much easier to be a “contented model defender”, merely following a methodology to-the-dot only because it worked in the past. The risk of this approach, though, is well summarized by Kieran Conboy[1]:
Rewarding compliance to a commercial agile method might provide an undesirable incentive for developers to adopt a ‘square peg in a round hole approach’, forcing these methods into situations to which they are not suited.
Becoming “optimistic model seekers"--i.e., examining the characteristics of the project at hand, and questioning assumptions regarding what will work given its particularities--, is what allows project teams to develop a much superior model, one that works best for the present situation (but needs to be reevaluated at the next project).
Too many software projects offer unsatisfactory outcomes, including projects using commercial agile frameworks. By understanding the flawed thinking behind the stance of the contented model defender, business analysts can opt instead to turn into optimistic model seekers, looking at each new software project as an opportunity to combine traditional and agile practices to establish a winning approach to delivering valuable software.
Author : Adriana Beal received her B.S. in electronic engineering and MBA in strategic management of information systems from two of the most prestigious graduate schools in Brazil. For the past 10 years she has been offering consulting assistance throughout the software development life cycle for organizations implementing large, complex software projects. Adriana recently joined e-commerce team at Hewlett-Packard. She is also the founder of Projeto 100%, a movement making significant changes in the lives of families trapped in poverty in Brazil.
[1] Conboy, Kieran. “Agility from First Principles: Reconstructing the Concept of Agility in Information Systems Development. Information systems Research Vol. 20, No. 3, September 2009, p. 332