Measurably Improving Your Requirements


Abstract. Requirements continue to be a major problem area for most organizations.  According to industry reports, the leading causes of quality, cost, and schedule problems are lack of understanding of the customer’s needs, incomplete requirement specifications, and managing changing requirements. In fact, requirements are so important that one of the definitions of quality is, “conformance to requirements”.  If requirements are not good, the costs of poor quality will be high and the resulting products and services will not be good either.  So what can an organization focus on now to measurably improve their requirements?  This article will describe some practical strategies that organizations can use to measurably improve their requirements.

Why Focus on Requirements?

 “The hardest single part of building a… system is deciding what to build...  No other part of the work so cripples the resulting system if done wrong.  No other part is more difficult to rectify later.”  [Brooks, Fredrick P., Jr.  “No Silver Bullet”.  IEEE Computer, 10-19, April 1987].  Many studies that are conducted to determine the top problems with projects come up with requirements at the top.  Many studies also estimate that about 60-70% of defects discovered during testing are due to inadequate requirements and design. Many reports on quality and delivery problems identify three leading requirements causes:

  • Lack of customer input

  • Incomplete requirements

  • Changing requirements

These requirement problems have been around a long time, and although things have improved a little over the years, these top requirement problems haven’t changed very much.  Requirements continue to be in the top 5 problems of most organizations. So what can an organization focus on now to measurably improve their requirements?

Measurably Improving Requirements

Requirement Metrics

This section describes some example successful metrics used at real organizations for measurably improving requirements:

  • Average effort per requirement (e.g., a productivity metric)

  • Average cost per requirement

  • Requirement cycle time

  • Requirement quality (e.g., requirement defects)

  • Requirement priority (e.g., can be set by customer)

  • Requirement volatility (e.g., a measure of the changing nature of the requirement)

  • Requirement stability (i.e., what is the probability of the requirement changing?)

  • Requirement risk (e.g., has the requirement been implemented before?)

  • Total number of requirements (e.g., a size metric)

Requirements are so important that they need to be measured, analyzed, and improved.  Requirements can be thought of as having “attributes” (e.g., priority, quality, risk, testable, etc).  Many of the attributes are measurable.  The requirement metrics and attributes can be tracked in a requirements tool or in a spreadsheet.  The requirement metrics are tracked over time so measurable improvement can be demonstrated.

LSI even uses requirements as a “size” and “productivity” metric in some leading edge organizations that has achieved measurable results.  Even “lean metrics” can be applied to requirements.  For example, cycle time for releasing requirements can be measured and improved.

Requirements also data needs to be analyzed for quality, productivity, and performance.  For example, scatter diagrams, histograms, run charts, Pareto charts, statistical process control charts (e.g., on defect density), etc., are a few of the data analysis tools that are useful for measurably improving requirements.

Requirement Defect Data

Interestingly enough, some of the most important improvement data comes from requirement defects.  If an organization does a good job of removing defects from requirements (very few organizations do a good job of this), they learn a lot about their requirements process from the types of requirement defects.  For example, one the most common defects types in requirements is “clarity”.  A Clarity defect means that a requirement is ambiguous (i.e., has more than one meaning) or it is vague (i.e., not clear).  One of the root causes of clarity defects is the English language itself.  The English language has many definitions for each word (causing ambiguity and vagueness).  Advanced solutions such as “operational definitions” can solve this problem.


Measurably improving requirements is not easy. Because requirements are primarily an intellectual activity, disciplined processes and metrics are necessary along with product knowledge and good people skills.  The author hopes that some of the practical metrics in this article will help your organization manage requirements more successfully, and make your requirements more measurable.  Lean Solutions Institute, Inc. (LSI) provides requirements training and consulting, and helps organizations to measurably improve their requirements.

Author: Mr. Timothy G. Olson is Founder and President of Lean Solutions Institute, Inc (LSI).

While performing quality consulting, Mr. Olson has helped numerous organizations measurably improve quality and productivity, define lean processes and procedures, save millions of dollars in costs of poor quality, and has helped organizations reach higher Software Engineering Institute (SEI) levels. Mr. Olson has been formally trained in Baldrige, Crosby, Deming, Juran, ISO, CMMI® and Six Sigma quality approaches.  Mr. Olson holds a Masters Degree in Computer Science from the University of Massachusetts, and was a lead-author of a Software Quality Course for the University of Minnesota Masters Degree Program in Computer Science.  He is a Juran Institute Associate, a senior member of ASQ, and a member of IEEE and NDIA.

Timothy G. Olson
Lean Solutions Institute, Inc. (LSI)
(760) 804-1405 (Office)
[email protected]

Copyright © 2010 Process Assets, LLC (PAL).

Like this article:
  13 members liked this article


Marc Thibault posted on Saturday, August 14, 2010 12:05 PM
I had expected a longer article as I started reading, but you put the case beautifully in very few words.

Bringing the value of information into the requirements discussion puts the focus where it most needs to be. Clear, unambiguous, comprehensive requirements are where a successful project starts. It's no guarantee of success, but the alternative guarantees failure.

zarfman posted on Monday, August 16, 2010 6:15 PM


Having spent many years in accounting and finance, I have questions regarding the following;

Olson wrote: Average effort per requirement (e.g., productivity metric).

Zarfman writes: Average, how is it computed and what is it compared against? What is met by effort, and how is it measured, perhaps in man hours?

Olson wrote: Average cost per requirement.

Zarfman writes: For what purpose? It appears to me that a rather complex and comprehensive record keeping system would be needed. Perhaps some form of standard cost system is required.

Olson wrote: Requirement cycle time

Zarfman writes: I don’t understand what type of cycle you are talking about.

Olson wrote: Requirement quality (e.g., requirement defects)

Zarfman writes: Who determines if a requirement is defective. Where does the decision maker derive his/her authority from?

Olson wrote: Requirement priority (e.g., can be set by customer)

Zarfman writes: Sounds reasonable to me.

Olson wrote: Requirement volatility (e.g., a measure of the changing nature of the requirement)

Zarfman writes: Volatility is a financial/accounting term with a very specific meaning. I’m not sure how one would measure requirement volatility. One could take the position that a requirement changes or it doesn’t. If a requirement changes once or twenty times what does that mean? The changes could be due to human error, further analysis or the result of litigation.

Olson wrote: Requirement stability (i.e., what is the probability of the requirement changing?)

Zarfman writes: How does on compute the probability of a requirement changing? Where does one get the necessary data for the probability computation?

Olson wrote: Requirement risk (e.g., has the requirement been implemented before?)

Zarfman writes: Do you mean with in the company or outside the company.

Olson wrote: Total number of requirements (e.g., a size metric)

Zarfman writes: What does this really mean? A project can have many requirements that are relatively simple. Or, very few requirements that are extremely complex.

In my experience when one uses the words average or probability they may have unintentionally wandered into the field of statistics. If one violates the necessary and sufficient conditions that must be met to compute a statistical average or probability then the computed value is erroneous.

Perhaps my understanding of your writings are incorrect. If so, let me know.


ColemanTO posted on Monday, March 21, 2011 4:05 PM
The ROI of User Experience with Dr. Susan Weinschenk:
Only registered users may post comments.



Copyright 2006-2023 by Modern Analyst Media LLC