A couple of years ago, I wrote a couple of related articles entitled “Verifying Requirements Documentation[1]” and “As a Project Sponsor, What is Your Risk Tolerance? [2]”
While the former provides a background on best practices, this is an update to the latter. It expands my proposed Risk Tolerance Model by including residual risk, a BRD evaluation scale, a BRD Risk Measurement Tool, possible BRD Risk Scenarios, and some wise words in the context of a quote from Voltaire on “The perfect is the enemy of the good.”
Risk Tolerance
Under the waterfall solution development life cycle (SDLC), the project sponsor signs-off on the business requirements document (BRD); the BRD being the synthesis of the requirements elicited during the analysis phase. What does sponsor sign-off mean? It means that the sponsor assumes the accountability (business responsibility) that the BRD reflects the business need. What motivates the sponsor to sign-off? It is when the BRD matches or surpasses the sponsor’s risk tolerance.
The sponsor’s risk tolerance is dependent on the characteristics of the company and the project. For example depending how deep the company’s pockets are (wealth), a failure of a $10M project could cause a company to go bankrupt, where that same company could easily survive the failure of a $1M project. Hence, when a sponsor signs-off on a BRD, it means the sponsor has weighed the thoroughness of the BRD and found the risk of defects acceptable.
The follow-up question and point of this article is “Can sponsors measure portions of the BRD in terms of various levels of defect[1] risk percentages?” These measures would help sponsors in the BRD sign-off decision.
An Expanded Risk Tolerance Model
Below is an expanded BRD Risk Tolerance model (Figure 1). It depicts a cumulative risk tolerance incline starting from a risk averse position (lower left corner) to a risk prone position (upper right corner). Supporting the risk incline are seven boxes representing best practice components for writing a BRD.
-
Under each best practice are key words describing the practice; for details see the reference, “Verifying Requirements Documentation[1].”
-
Within each box is the range of risk assumed when the component is missing. Note that the minimum risk is 1% with a maximum percentage for that component.
-
On top of each best practice is a cumulative risk tolerance percentage. This percentage represents the cumulative risk the project sponsor assumes when BRD components are missing.
-
The maximum risk is 93%; this includes the maximum risk for each best practice missing plus a general residual risk of 3%.
-
The minimum risk is 10%; this includes a 1% minimum for each best practice plus a general residual risk of 3%.
-
For example, a risk tolerance of 36% equates to signing-off on a BRD with all the best practice components completed except: traceability (5%), data (5%), transitional requirements (10%), nonfunctional requirements (10%), residual (3%) plus the minimum 1% for each best practice component completed (Business Rules, Functional Requirements and Process Improvement).
Figure 1. Updated Business Requirements Document Risk Tolerance Model
Note that I have based the risk tolerance percentages on the overall BRD components. Your perspective on the risk tolerance percentages may be different due to your involvement on a project. For instance, if you are part of the infrastructure, your concern with nonfunctional may be higher.
Risk Evaluation
To augment the BRD Tolerance Model, I developed a simple tool for evaluating each best practice based on BRD expectations[1]; see Figure 2. The tool uses a scale rating system between 0-5; I feel this narrow granularity is adequate and still provides some differential range. The sponsor judges each BRD component using the scale, which equates to a component risk percentage. Note the tool provides the total risk if the sponsor selects a single rating across all the components. For example, if the sponsor rates all the components 2 (does not meet expectation), then the sponsor assumes a 64% total risk; if the sponsor rates all the components 3 (meets lower expectation), then the sponsor assumes a 46% total risk.
Note: you might consider a rating of 3 to be an average risk expectation. However, if you rate all the components 3, the total risk would be an uncomfortable 46%. This is why I consider a rating of 3 meeting a lower expectation rather than average.
I have recommended a “sweet spot” on total risk – somewhere between 28-10 percent. Per the model, 10% is the lowest risk achievable. Of course, the component rating can vary per the judgment of the sponsor. See Figures 3 and 4 for different scenarios in table and radar plot format.
Figure 2. BRD Evaluation Tool
Figure 3. Possible BRD Risk Scenarios
Figure 4. BRD Risk Radar Plot Example
Note that the radar plot is just another format that graphically shows a risk rating. We use radar plots when we need to evaluate various aspects of a subject, for example, evaluating multiple aspects of an agile project or an agile team. My understanding is that coaches use radar plots to evaluate athlete performance such as the NFL scouting combine.
First a Note on Project Risk
Before reading the below examples, note the percentages on the model are the sponsor’s risk tolerance for a project equated to “allowed” missing BRD components. As stated before, project risk depends on the characteristics of the project (i.e., size, complexity, cost/benefits, technology, resources, duration, etc.). The point here is that the project risk may well influence the project sponsor on what is an acceptable BRD.
Risk Tolerance Model Examples
Below are examples of using the Risk Tolerance Model on evaluating a BRD.
-
A project sponsor justifies a high-risk tolerance with the BRD missing various components for a project entailing small modifications to an existing system (i.e., the cost of the BRD should not exceed the benefit of the modification). See Figure 3, possible scenarios 7 & 8.
-
A project sponsor justifies a low-risk tolerance of missing any components in the BRD for a project providing a strategic new service for the business (i.e., product defects will have a major impact to the business). See Figure 3, possible scenarios 1 & 2.
-
A project sponsor concerned with the project schedule issues decides to accept a BRD without a business rules component. See Figure 3, possible scenario 4.
Use of the BRD Risk Tolerance Model
The business analyst can use this model to provide clarity in deciding that the BRD is “done” in BRD verification and validation meetings with BA peers and stakeholder groups respectively. The model helps in guiding sponsors in approving a BRD. Project sponsors have asked me many times how do I decide to sign-off on the BRD: “Is there a magic correlation between the number of pages and project cost?” No such luck. The answer is in the following question, “What is your risk tolerance?” Using the BRD Risk Tolerance Model, the sponsor can rate each of the BRD components to determine the associated risk.
Can the Product Owner (sponsor) use the BRD Risk Tolerance Model on an Agile (scrum) Project?
Since agile software developers do not write a formal BRD, the answer is literally, no. However, in principle, the answer is yes. Whether the SDLC is waterfall or agile, developers still need to determine requirements (functional, nonfunctional, and transitional), still need to understand business rules, still need to determine the data to be used, and still need to stay within scope (traceability). However, since agile is a software development SDLC, not business process improvement, one must look to Lean / Six Sigma or some other methodology. Note that agile does include software development improvement (retrospective), but that is different from business process improvement
The product owner (sponsor) still needs to consider risk tolerance when reviewing the product backlog and during each retrospective prior to approving the product release. Questions to consider:
-
Are specific functional requirements present? Is the developing team addressing them in business value order? Are dependencies considered?
-
Are nonfunctional requirements measured?
-
Has the developing team incorporated the business rules within the software decisions, access authorizations, and conditions?
-
Are data relationships and cardinalities present in the files?
-
If replacing a legacy system, has the developing team addressed any needed transition requirements?
-
Does the product backlog reflect the original scope features?
Summary
The question this article set out to address was “Can sponsors measure portions of the BRD in terms of various levels of defect risk percentages?” Yes, they can. Perhaps you prefer different components, a more granular rating system, or different risk percentages for the components; I invite your comments. As I mentioned in Figure 2, the component risk percentages reflect my experience – not a definitive study.
In the final analysis, the project sponsor decides to approve or not approve the BRD. But, this is only the position of yes or no. What were the criteria used to form that position? The Risk Tolerance Model helps form the position. As far as my “sweet spot” on acceptable risk, let me close with a quote from Voltaire (1694 -1778), the French writer, philosopher, and playwright.
Author: Mr. Monteleone holds a B.S. in physics and an M.S. in computing science from Texas A&M University. He is certified as a Project Management Professional (PMP®) by the Project Management Institute (PMI®), a Certified Business Analysis Professional (CBAP®) by the International Institute of Business Analysis (IIBA®), a Certified Scrum Master (CSM) and Certified Scrum Product Owner (CSPO) by the Scrum Alliance, and certified in BPMN by BPM essentials. He holds an Advanced Master’s Certificate in Project Management (GWCPM®) and a Business Analyst Certification (GWCBA®) from George Washington University School of Business. Mark is also a member of the Association for the Advancement of Cost Engineering (AACE) International, and the International Association of Facilitators (IAF).
Mark is the President of Monteleone Consulting, LLC and author of the book, The 20 Minute Business Analyst: a collection of short articles, humorous stories, and quick reference cards for the busy analyst; available on amazon.com and extended networks. He can be contacted via – www.baquickref.com.
References
-
Monteleone, Mark (2012), “Verifying Requirements Documentation.” This article provides a checklist of best practices on verifying documented requirements.”
-
Monteleone, Mark (2012), “As a Project Sponsor, What is your Risk Tolerance?” This article proposes a risk tolerance model as a sliding scale based on what is included in the contents of the BRD.