I have spent the last year and a half working on an enterprise software solution development effort where we do not use a Requirements Management tool like Caliber or Visual Studio TFS. Our requirements are created in Word using standardized templates and distributed to Development and Test teams for consumption.
Test cases are written in Excel and tied to the requirements in the documents. In general, I would have to say that coverage is good but not complete (I know this anecdotally since there is no good way using Excel and a bunch of Word documents to know for certain). In theory, a failed test case should mean that a requirement is not satisfied and pinpoints a missed feature or requirement.
This system breaks down totally when it comes to Change Requests that are created during the course of the project. Change Requests are entered directly into a defect tracking system. Change Requests are usually supposed to have detailed requirements associated with them but in practice the quality of the supporting documentation has varied widely. So, Change Requests have little to no systematic traceability associated with them. This is not to imply that the Change Requests are poorly implemented. Just that doing any kind of systematic tracing exercise against them is near impossible.
The key problems I have found with using Excel to perform traceability are as follows.
1. Forward traceability from Business Objectives or High Level Features to specific requirements is very difficult to do and in many cases is just not practical.
2. Managing requirements as they change is very difficult to do. You could have false positives where the spreadsheet tells you there is good coverage without realizing that the underlying requirement itself has changed.
3. Managing multiple tests for a single requirement because very difficult. For example, if a single requirement has to pass 3 test cases for it to be considered fully implemented, the spreadsheet approach becomes error prone and hard to understand very quickly.
4. The spreadsheets themselves become unwieldy as multiple requirements and tests are entered. The volume of data becomes hard to manage and consume.
5. Reporting becomes a hit and miss process. It requires a lot of manual effort, is time consuming and error prone.
6. Requirements that do not start life in a requirements document (Change Requests) are seldom tracked as rigorously as standard requirements.
7. Historical analysis is very difficult to do. On projects that last several years, digging up an old Excel spreadsheet to determine if specific requirements were implemented or not a year ago can easily become a week long exercise in futility.
So what then is the answer to the question I posed at the beginning?
Excel is fine for small projects but larger enterprise grade efforts require a specialized requirements tool with good tracing features.
The odds of performing good traceability on your project are significantly improved when using a requirements management tool. There are real costs associated with unimplemented or improperly implemented requirements. A good tool gives you a better chance of catching these kinds of errors with good traceability features. So, when considering a tool to manage your requirements, do not overlook the quality of their traceability features.
For more check out our blog: http://requirements.seilevel.com/blog/
By Abadri