If someone said you could only perform a single quality practice on a software project, what would you choose? I’d pick peer reviews of requirements, which I believe are the highest-leverage quality practice we have available today.
In a peer review, someone other than the author of a work product examines the product for quality problems and improvement opportunities. Reviewing requirements is a powerful technique. Use them to identify ambiguous or unverifiable requirements, find requirements that aren’t sufficiently detailed yet, spot conflicts between requirements, and reveal numerous other problems.
The most effective type of peer review is a structured process called an inspection. Inspection of requirements is one of the highest-leverage software quality techniques available. Several companies reported that they avoided up to ten hours of labor for every hour they invested in inspecting requirements documents and other software deliverables. Who wouldn’t want to try a technique that might offer a 1,000 percent return on investment?
A peer review is both a technical activity and a social interaction. Asking some colleagues to tell you what’s wrong with your work is a learned—not instinctive—behavior. It takes time for a software organization to instill peer reviews into its culture. This article, adapted from my book (with Joy Beatty) Software Requirements, 3rd Edition, points out some common challenges when performing requirements reviews, along with suggestions for how to address each one. For more information about conducting reviews and inspections, see the book Peer Reviews in Software: A Practical Guide.
Large Requirements Documents
The prospect of thoroughly examining a long requirements document is daunting. You might be tempted to skip the review entirely and just proceed with construction, trusting the requirements to be correct and complete—not a wise choice. Even given a document of moderate size, all reviewers might carefully examine the first part and a few stalwarts will study the middle, but probably no one will look at the last part.
To avoid overwhelming the review team, perform incremental reviews throughout requirements development. On agile projects, spend some time reviewing requirements during each iteration. Even if you’re using informal requirements approaches, it’s always a good idea to put some heads together to confirm understanding, look for hidden assumptions, and ensure all exceptions have been addressed. Use inspections to take a close look at high-risk areas, and use informal reviews for less risky material. You might ask certain reviewers to start at different locations in the requirements set to make certain that fresh eyes have looked at every portion.
To judge whether you really need to inspect the entire requirements set, examine a representative sample carefully. The number and types of errors you find will help you determine whether investing in a full inspection is likely to pay off. Of course, even if your sample looks great, you don’t know for sure what issues might be hiding in the remainder, so there’s still some risk there.
Large Review Teams
Many project participants and customers hold a stake in the requirements, so you might have a long list of potential participants for requirements reviews. However, large review teams increase the cost of the review, make it hard to schedule meetings, and have difficulty reaching agreement on issues. I once participated in a meeting with thirteen other reviewers. Fourteen people cannot agree to leave a burning room, let alone agree on whether or not a particular requirement is correct. Try the following approaches to deal with a potentially large inspection team:
- Make sure each participant is there to find defects, not to be educated or to protect a political position.
- Understand which perspective (such as user, developer, or tester) each inspector represents. Several people who represent the same community can pool their input and send just one representative to the inspection meeting.
- Establish several small teams to inspect the requirements in parallel and combine their defect lists, removing any duplicates. Several studies have shown that multiple inspection teams find more requirements defects than does a single large group. The results of such parallel inspections are primarily additive rather than redundant.
- Convene a small inspection team of four to seven individuals, but supply the requirements set to the other interested stakeholders in advance so they have an opportunity to contribute their input.
Geographically Separated Reviewers
Geographically dispersed teams often collaborate to build products these days. This makes reviews more challenging. Teleconferencing doesn’t reveal the body language and expressions of other reviewers like a face-to-face meeting does, but videoconferencing can be an effective solution. Web conferencing tools allow reviewers to ensure that they are all looking at the same material during the discussion.
Reviews of an electronic document placed in a shared network repository provide an alternative to a traditional review meeting. In this asynchronous review approach, reviewers use word-processor features to insert their comments into the text. Each comment is labeled with the reviewer’s initials, and each reviewer can see what previous reviewers had to say. Web-based collaboration tools also can help. Some requirements management tools include components to facilitate distributed asynchronous reviews that do not involve live meetings.
If you choose not to hold a meeting, recognize that this can reduce a review’s effectiveness. Various studies have shown that synergy and discussion among the participants in a review meeting often triggers thoughts that discover defects none of the individual reviewers spotted during their individual preparation. An asynchronous review is certainly better than performing no review at all, though. Just don’t let debates in the form of word-processor comments substitute for talking to each other.
Collaborations across distance often involve people who work in different cultures, sometimes different countries. Different cultures have varying attitudes towards reviews. For instance, people in some cultures find it highly uncomfortable to make critical observations about someone else's work. As a result, they might not say anything during a review meeting. This is a good way to avoid hurting someone's feelings, but not a good way to make a work product better. Be aware of such cultural differences, and try to create safe environments (such as meeting-less asynchronous reviews) for people to raise potential defects without making anyone uncomfortable.
Unprepared Reviewers
A prerequisite for a formal review meeting is that the participants have examined the material being reviewed ahead of time, identifying the initial issues they want to raise. Without this preparation, you risk people spending the meeting time doing all of their thinking on the spot and likely missing many important issues. In fact, if you’re invited to participate in a requirements review and don’t have adequate time to go over the material in advance on your own, don’t even bother attending the meeting. It’s a waste of everyone’s time.
One project had a 50-page requirements spec to be reviewed by 15 people (far too large a group to be effective and efficient). The reviewers were given one week to examine the document on their own and send issues back to the author. Not surprisingly, most people didn’t look at it at all. So the lead business analyst scheduled a mandatory meeting for the inspectors to go through the document together.
The BA projected the SRS on the screen, dimmed the lights, and began reading through the requirements one by one. (The room had one very bright light shining down in the middle, directly on the lead BA—talk about being in the spotlight!) A couple of hours into the review meeting, the participants were yawning, their attention fading. Not surprisingly, the rate of issue detection decreased. Everyone longed for the meeting to end. This BA finally let the participants leave, suggesting they review the document on their own time to speed up the next review meeting. Sure enough, being bored during the meeting triggered them to do their prep work the next time.
Make Reviews Work for You
Reviewing requirements is not everyone’s idea of a fun time. Peer reviews can be tedious and time consuming, sometimes even stressful. Nonetheless, if you’re serious about maximizing the quality of your software, your teams will review all of their requirements, using the more formal inspections to carefully examine the most critical or error-prone portions. My general rule is: “Review early and often, formally and informally.”
The teams I know that have adopted requirements inspections agree that every minute they spent was worthwhile. Use the suggestions in this article to help your teams maximize the return on investment they make in requirements reviews.
Author: Karl Wiegers & Joy Beatty
Karl Wiegers is Principal Consultant at Process Impact, www.processimpact.com. Joy Beatty is a Vice President at Seilevel, www.seilevel.com. Karl and Joy Beatty are co-authors of the award-winning book Software Requirements, 3rd Edition, from which this article is adapted. Karl’s most recent book is Successful Business Analysis Consulting: Strategies and Tips for Going It Alone.