Chapter 19 Ambiguity Metrics
Continuing the the series of samples from various books, this week I'm posting a chapter from Exploring Requirments Two, just posted to Leanpub.com as a bargain bundle with Exploring Requirements One. - See more at: http://secretsofconsulting.blogspot.com/#sthash.lSTCEoew.dpuf
Contrary to what some people think, requirements do have to be tested. At least, they must be tested if the project is to have a fighting chance of success. Part V of Exploring Requirements Two, develo specific ways in which requirements can be tested.
19.1 Measuring Ambiguity
The whole purpose of the requirements process is to reduce ambiguity in the development process, so the most basic test of any requirement is to measure its ambiguity. We've discussed requirements ambiguity, but the term "ambiguity" is itself ambiguous. There are many ways to reduce the ambiguity in "ambiguity," but the best would be to specify a precise way of measuring it. After all, there's little ambiguity in this requirement:
A. Draw a straight line on a blank sheet of 8.5" x 11" white paper, 13.150 ± .025 centimeters in length, .500 ± .025 centimeters in width, using a freshly sharpened Dixon Ticonderoga 1388 number 2 pencil. The line should be parallel to the top of the page, 2.000 ± .025 centimeters from the top, and touching the unbound edge.
On the other hand, we've already seen how much ambiguity there can be in a requirement that says:
B. Design a transportation device.
If "ambiguity" is a property of a requirement, then we'd like to be able to measure it in such a way that statement A has a small amount and statement B has a large amount.
19.1.1 Using the ambiguity poll
The ambiguity measure we will develop is based on something we already observed in the "Design a transportation device" example. We gave statement B to a thousand individuals, instructing them to work independently on a solution. As we saw, this problem statement lacks many essential ingredients, and we could have anticipated that each participant would resolve each one uniquely. Even if all solvers were trained to use the same design process, we'd be shocked to find that even two had produced exactly the same design.
Now suppose we gave statement A to a thousand individuals, working independently on a solution. The problem statement lacks a few essential ingredients, and since people are different, we would expect some individuals would create idiosyncratic solutions. Generally speaking, though, we'd expect there would be only a few variations, and most solutions would be indistinguishable.
This mental experiment suggests we can measure ambiguity as the diversity of interpretation. We have actually performed this experiment with large groups. For problem A, we measured the diversity by comparing the lines drawn. Everyone drew a single line, in pencil, and there were only minor variations in line length and placement on the page.
For the transportation problem (B), we gave each person one minute to develop a conceptual solution. At the end of that time, we asked each to give a single number as an estimate of the manufacturing cost of the proposed transportation device. Obviously, if they all had designed the same solution, their manufacturing costs would have varied somewhat, but not by much. In fact, for roughly a thousand people, the cost estimates varied from $10 to $1.5 billion. This ratio of 1,500,000,000/1 indicates an extreme difference of opinion regarding the meaning of the problem, but isn't that precisely what we mean by ambiguity?
Such a ratio of largest to smallest estimate in a poll of informed individuals can be used as an ambiguity metric, a measurable entity for which we can obtain a precise value. Of course, a precise metric may not be extremely accurate, but it's far better than no measure at all. Indeed, it's a rather practical measure, one we have often used in real design situations.
19.1.2 Applying the polling method
A manufacturer of precision electrical components was studying the feasibility of developing an automated system for handling catalog information requests. The company's president had cost estimates ranging from "moderately inexpensive" to "moderately expensive," and didn't know whether to authorize the project. We asked each of 14 qualified individuals to write an independent estimate of the project cost. The estimated amounts ranged from $15,000 to $3 million. When the company president saw this 200/1 ratio, he halted the feasibility study to wait for a less equivocal requirement.
19.1.3 Polling on different bases
"Manufacturing cost" is only one possible basis for an ambiguity poll. Others that come quickly to mind are
1. total design and development cost
2. worker-years required for design and development
3. number of unique parts in the solution
4. minimum calendar time required to deliver the first product
These measures would apply to any project, but others would be more specific. For instance, in the transportation device problem, we might ask for estimates of
5. efficiency: average energy consumption per hour of use
6. range: how far it could transport
7. capacity: maximum weight it could carry at one time
19.2 Using the Metric as a Test
One useful model of design says design is the process of removing ambiguity. In terms of this model, design proceeds through a series of steps: creating an approximate design, testing for ambiguity, removing the ambiguity found, and retesting the new approximation. Eventually, the tests say the latest approximation is close enough, and design stops.
(In the book, the text continues to explain in detail how to use the ambiguity metric as a test in several different ways.)
BARGAIN BUNDLES
For the end of this year, I'm offering a list of bargain book bundles from Leanpub.com. Take a look at the books in each bundle by clicking one of these links:
(Get both Volume One and Volume Two in a single bargain bundle)
No comments:
Post a Comment