The IIBA (International Institute of Business Analysis), in collaboration with the Agile Alliance, recently released a draft for public review of the Agile Extension to the BABOK®. The guide describes “business analysis areas of knowledge, their associated activities and tasks, and the skills necessary to be effective in their execution within the framework of agile software development”.
Below are 3 misconceptions that, in my opinion, the current draft of the Agile Extension is helping perpetuate:
1. There are only two models, agile and waterfall. If you aren’t using the former, you must be following an ineffective plan-driven methodology with a single requirements phase.
The Agile Extension briefly recognizes the existence of non-agile, non-waterfall methodologies, stating, on page 7, that “agile methodologies are a subset of iterative software development”. However, other parts of the document are written in a way that reinforces the prevalent misconception that the only alternative to agile is the failure-prone waterfall method. I’ve extracted two examples to illustrate this issue:
“Analysts are required to approach requirements from a 360 degree perspective. They are required to work with the business sponsor on a strategic level, and define how the proposed product or feature aligns with the organizations portfolio and strategy. They must then work with the business and project team to break this vision down into requirements that support effective and accurate estimation. In an agile project this is done for each iterative release, as opposed to the single requirements phase that exists in plan‐driven methodologies. The analyst delivers just‐in‐time, detailed requirements to the development team so they can build only what is required for a specific iteration.” (p. 5) [emphasis mine]
There are different ways of detailing requirements in plan-driven methodologies that are neither just-in-time nor the monolithic requirements phase typical of the waterfall model. I know many BAs (myself included) who have been performing the exact same tasks described above in non-agile environments, avoiding both extremes of just-in-time requirements and requirements detailed in their entirety prior to the development phase. Many of the methodologies I’ve used in projects in the past 15 years could be classified as “plan-driven”, but rather than proposing a single requirements phase, they prescribe overlapping iterative development cycles in which requirements and design are refined and revised during project execution.
The concept that there are alternatives to the opposing models of JIT and the “big requirements upfront approach” should not be ignored in a guide about agile for business analysts. Such understanding is crucial to allow analysts the opportunity to evaluate the different alternatives in requirements processes that are available to support agile projects with different characteristics and levels of complexity.
“Agile offers the opportunity for business analysis to benefit from the frequent feedback provided by the business. “ (p. 3)
Again, the implied assumption here is that, outside of agile, you are working on a sequential, waterfall model that prevents feedback from happening throughout the course of the project. In the plan-driven, non-agile development efforts I’ve worked on, early and frequent feedback for the analysis work was always available, either as part of a formal process, or in response to an informal request from the BA to the business stakeholders. Such feedback loops were available throughout the stages of requirements discovery, validation, and communication. It’s true that agile methods facilitate frequent feedback, but there is nothing preventing non-agile development approaches from obtaining frequent feedback from business stakeholders, not only for written requirements, but also working software, early models created to test the concepts of the system, or even paper prototypes.
2. The model of just-in-time requirements is superior to all others
“By providing just‐in‐time requirements, there is less rework of requirements because only the requirements required for the current release are defined in detail and developed.” (p. 3)
I don’t see, in real life, any supporting evidence to the conclusion that just-in-time requirements necessarily translate into less rework. Sure, if you prepare requirements 2 years, or even 2 months, before development starts, there is a high probability that changes will be needed in the original requirements to reflect evolving business needs. On the other hand, producing subsets of requirements focused on just what is required for the current release in some cases significantly increases the risks of important requirements being overlooked or misunderstood. Limiting the analysis to what is being produced in the next cycle often prevents analysts from studying requirements in relationship to other requirements, and from seeing each piece of a system as a meaningful component of a working whole. An example of just-in-time requirements leading to substantial amount of rework due to lack of systems thinking can be found in this article: Agile vs. Traditional Software Development: Why is the debate still going on? - Part I: The dangers of bounded rationality.
3. Documentation should be automated, barely sufficient to meet the needs of the team, and written to fulfill an immediate need
“A general principle with agile is that if a document is valuable enough to be created, it is probably important enough to be automated so that it is part of the living code base. This has lead to the rise of automated acceptance testing and behaviour driven development that allows business analysts to work with quality assurance professionals to create executable specification in the form of examples.
This principle comes directly from the Agile Manifesto which says ‘Working software over extensive documentation’. It is often misinterpreted as meaning no documentation. Instead, documentation should be barely sufficient to meet the needs of the team.” (p.100)
“In agile methodologies, the software is constructed within days of the team agreeing to deliver a set of requirements, and so does not need to include information that the team can remember or agree to during the construction process.” (p.99)
“Ensure that all documentation produced through business analysis is intended to fulfill an immediate need, has clear value for stakeholders, and does not create unnecessary overhead.” (p.99)
“The context plays an important factor in the amount of documentation required. Some projects are mandated to produce documentation by external entities (for example, regulators). Documentation may also be needed to provide a long‐term record of decisions reached by the team or functions implemented in the application. However, this documentation can be written after the software is developed, which ensures that it actually matches what the team delivered.” (p. 100) [emphasis added]
I am yet to find data to back the claim that automated, barely sufficient documentation produced to fulfill an immediate need generates better, faster, or cheaper results in most software projects. The agile concept of “just enough, barely sufficient” documentation is an understandable reaction from the software development community to the exorbitant amount of paperwork often required in software projects (mainly in large companies). However, in my consulting practice I’ve seen many organizations suffer the consequences of waiting to document a complex software product after it was developed, or, even worse, just using code and automated tests as the only documentation of system behavior, and postponing additional documentation until someone asks for it.
A typical example was a project implementing complex business calculations that used multiple variables in various combinations. The developers’ work was based mainly in face-to-face conversations, with minimal documentation. After deployment, remembering exactly what was decided for each possible combination of variables was tricky, even for skilled developers who had worked on this part of the solution. Developers found it hard to determine in detail what the system was doing in some of the exceptional cases, and to remember why the system was made to behave like it did. The time required to recover such knowledge (including the background of some decisions that were not captured anywhere) was significant. The impact included avoidable costs reconstructing the information, the refusal from some customers to use the system until a specification was produced, and uncertainty and delays during the discussion of enhancements to the product. In this case, and in many others in my past experience, producing just enough documentation to allow the software to be built may have generated short-term benefits, but at a high cost to future decision-making and system adoption rates. I see a level of incoherence between the agile concepts of “embracing change” and “barely sufficient documentation”, as the latter often proves to be a hindrance to future change, as people struggle to remember, or are no longer available to share, undocumented implementation details, and the reasons behind them.
The idea of avoiding excessive documentation is clearly beneficial to software projects. Still, while it’s a positive development to get rid of documentation created just to “check a box”, or to ensure that the performance of a team can be measured against some bureaucratic rule, it seems dangerous to start promoting, as a general practice, barely sufficient, preferably automated, just-in-time documentation. Where is the evidence supporting this approach? Perhaps it’s time that we BAs start following Greg Wilson’s suggestion, and adopt higher standards of proof before a concept is accepted as a general recommendation.
- - -
The Agile Extension to the BABOK represents an important contribution to the business analysis community, offering analysts a useful introduction to agile concepts and explanation of the typical working practices used by BAs in agile projects. The document also provides an overview of some of the commercially available agile approaches, recommends BAs to research various agile frameworks to understand their pros and cons, and recognizes that it is “common for project teams to blend characteristics from more than one agile methodology based on unique team composition, skills, (...) and other factors”.
As stated in the guide, many of the concepts and approaches described will prove valuable to business analysis in any methodology and environment. The missing piece, in my opinion, is a clearer acknowledgement that:
1) waterfall is not the only alternative to agile frameworks--there are other models that are being successfully used in IT projects throughout the world, from which BAs can learn and benefit;
2) agile practices (such as JIT requirements) won’t necessarily produce superior results in all projects;
3) avoiding excessive documentation, and any documentation that doesn’t serve a valuable purpose, is a valid goal, but that doesn’t mean that automated documentation, or documents prepared only to fulfill an immediate need, are good replacements to quality documentation that may have to be written long before its needed, as means to ensure the right information about system behavior will be available when it becomes necessary for the organization to retrieve it.
As I wrote in another article,
Agile practices are not intrinsically “value-adding” — they must be aligned to business needs and goals in order to provide true value. Customary practices such as on-site customer, stand-up meetings, or test-driven programming may be suitable for some projects, but should not automatically be deemed supportive of agility just because they are part of a commercially available agile method.
In addition to describing agile practices, and recommending that analysts spend time researching the various methodologies to understand their pros and cons, the Agile Extension should encourage BAs to use their analytical skills, and influence within their project team, to promote the selective borrowing from agile and traditional methods to fit the needs of each individual project. Many of the agile teams I’ve worked with recently have been developing “hybrid models” that incorporate elements from agile and more traditional software development methods in order to increase the odds of the solution delivering real value for stakeholders. A good example is the initiative to identify areas for which producing detailed requirements upfront can mitigate project risks caused by a lack of understanding of the impact a component will have in other parts of the system. Another is the use of comprehensive documentation to ensure compliance with regulatory requirements, while still taking advantage of agile practices such as short development cycles that continuously deliver working software.
By recognizing that many non-agile practices are also capable of producing business value, and that successful results may require blending elements from agile and traditional models to better fit project needs, the Agile Extension would help more teams achieve the benefits of agility while avoiding the “square peg in a round hole” syndrome that often forces agile methods into situations to which they are not suited.
Adriana Beal is a native of Brazil living in the U.S. since 2004. For the past 15 years she has been leading requirements discovery efforts and software process improvement initiatives for a diverse client base that includes IT, telecom, and major U.S. institutions. Adriana specializes in business analysis of complex software systems, and frequently uses hybrid software development approaches that combine agile and traditional practices to ensure the product’s strategic alignment to the business need. She has two technical books published in Brazil, and work internationally published by IEEE and IGI Global. Her educational background includes a B.S. in Electrical Engineering and an MBA with emphasis on strategic management of information.