See this Amp at http://amplify.com/u/k4ix
Thursday, December 30, 2010
Resolution #2 for the new year
Not just by writing, do more to help humanize computer business–a long way to go: <http://www.smashwords.com/profile/view/JerryWeinberg>
Resolution #1 for the new year
New Year Resolution #1: Make all my writings available in all eBook formats, so I'm free to write some more novels, stories & non-fiction. <http://www.smashwords.com/profile/view/JerryWeinberg>
See this Amp at http://amplify.com/u/k4hx
Friday, December 24, 2010
n Gratitude of Gratefulness
Amplify’d from www.tommyangelo.com
6 Comments
In Gratitude of Gratefulness
Posted by: Tommy Angelo on December 24th, 2010
“Finish your food! Think of the starving children in China!”
That was a typical thing for parents like mine to say to kids like me as I poked at the mucoid vegetables on my plate.
“Think of the starving children in China.”
That saying failed utterly at its purpose. All it did was make me resent the alleged starving Chinese children as much as I resented being forced to eat snot. And the resentment was just getting warmed up. For the next few decades, when someone said something about how I should be grateful or thankful or whatever, I resented them for even suggesting such a thing. First, I always had lots of problems: work problems, money problems, car problems, friend problems, lover (or lack of lover) problems, etc. I kept track of and organized my problems. You want me to be thankful? Have you seen my list of problems lately?
See this Amp at http://amplify.com/u/jfcz
The Parable of the Ones
This little essay on the consequences of choosing the wrong measurements is taken from my book, How to Observe Software SystemsOne way to avoid missing important observations is to observe everything. This strategy is neither humanly possible nor economically feasible. Resources devoted to observing one thing detract from the resources available to observe other things. Perhaps that's why some Routine managers are so fond of huge "metrics" programs—as a distraction from the things they really should be observing, but don't know how to observe without being overwhelmed.
Consider the Parable of the Ones:
Merlin the Manager was tired of being chastised by his boss, Wanda, for low programmer productivity. "How can I show you that the programmers are doing something," he asked her, "when all they're ultimately producing is ones and zeros."
"I'm not interested in zeros," Wanda complained. "Zeros are nothing. How many ones are they producing?"
"Um, I don't know," Merlin stammered.
"Well, you're their manager," Wanda accused. "You should know."
"Of course," Merlin apologized, backing out of Wanda's office. "I'll institute a metrics program."
Merlin then hired some measurement consultants who showed him how to count the ones automatically in every object program, plotting them by project and programmer. The initial report showed an overall productivity of 43.78% ones, and Merlin called a meeting of all the programmers to chastise them about their low productivity.
"Look at this figure," he accused. "This means that 56.22% of all bits on memory are essentially unused—filled with zeros. Why, when I was a programmer, I could generate programs at random that were 50% ones. If this keeps up, there won't be any performance awards this year, I can assure you."
Two months later, just before the performance awards were decided, Merlin looked at his metric report and was delighted to discover that the overall productivity figure was 53.04% ones. He showed this report to Wanda, who gave him a big bonus. "Well," he thought, "that certainly shows the value of a measurement program. Now, as soon as I fire those two programmers with less than 45% ones, productivity will show another boost."
Merlin, of course, was merely illustrating DeMarco's principle. If he rewards programmers for ones, he'll get ones. Even without explicit rewards and punishments, the mere fact of observing something implicitly reinforces it. Workers always notice what their managers notice—and what they fail to notice.
As often happens, Merlin's "measurement" system has produced a backwards effect from what the organization really wants (see diagram of effects on left). If he interprets the production of zeros as wasted effort, he encourages programmers to put effort into producing ones, which really is wasted effort. Much of the effort that might have gone into correct, well-performing software has gone instead into beating the measurement system.
Thursday, December 09, 2010
Testing Without Testing
The job of software testing, we know, is to provide information about the quality of a product. Many people, however, believe that the only way to get such information is to execute the software on computers, or at least review code. But such a belief is extremely limiting. Why? Because there's always lots of other information about product quality just lying around for the taking - if it were only recognized as relevant information.
Because of their psychological distance from the problems, external consultants are often able to see information that escapes the eyes and ears of their clients. Dani (Jerry's wife) tells this story about one of her consulting triumphs in the dog world:
======================================================
A woman with a Sheltie puppy was at her wits' end because "he always poops on the living room rug." She loved the little guy, so before giving up and taking him to the Humane Society, she came to Dani for a consultation. Dani listened to the woman describe the problem, then asked, "Are there any other behavior problems you've noticed?"
The woman thought about that for a while, then said, "Well, yes, there is one. He has this annoying habit of scratching on the front door and whining."
======================================================
It's easy to laugh at this woman's inability to see the connection between the two "problems," but that sort of blindness is typical of people who are too close, too emotionally involved, with a situation. Learning to recognize such free information is one of the secrets of successful testing. Using it, you can learn quickly about the quality of an organization's products or the quality of their information obtained from machine testing.
Here are some "Sheltie stories" from our consulting practices. Test yourself by seeing what information you can derive from them about the quality of a product or the quality of the information that's been obtained about the product through testing:
======================================================
1. Jerry is asked to help an organization assess its development processes, including testing. Jerry asks the test manager, "Do you use specs to test against?"
Test manager replies, "Yes, we certainly do."
Jerry: "May I see them?"
Test manager: "Sure, but we don’t know where they are."
======================================================
2. Jerry is called in to help a product development organization's testing group. He learns that they are testing a product that has about 40,000 lines of code.
"The problem," says the test manager, "is with our bug database."
"What's wrong with it," Jerry asks. "Is it buggy?"
"No, it's very reliable, but once it holds more than about 14,000 bug reports, its performance starts to degrade very rapidly. We need you to show us how to improve the performance so we can handle more bug reports."
======================================================
3. Bunny is asked to help improve the testing of a large product that has 22 components. The client identifies "the worst three components" by the high number of bugs found per line of code. Bunny asks which are the best components and is given the very low bugs per line of code figures for each.
She then examines the configuration management logs and discovers that for each of these three "best" components, more than 70 per cent of their code has not yet successfully been checked in for a build.
======================================================
4. Trudy is invited to help a development manager evaluate his testing department's work. She starts by looking at the reports he receives on the number and severity of bugs found each week. The reports are signed by the test manager, but Trudy notices that the severity counts have been whited out and written over.
"Those are corrections by the product development manager," the development manager explains. "Sometimes the testers don't assign the proper severity, so the development manager corrects them."
Using her fingernail, Trudy scratches off the whiteout. Under each highest severity count is a higher printed number. Under each lowest severity count is a lower printed number.
======================================================
5. Jerry is watching a tester running tests on one component of a software product. As the tester is navigating to the target component, Jerry notices an error in one of the menus. The tester curses and navigates around the error by using a direct branch.
Jerry compliments the tester on his ingenuity and resourcefulness, then asks how the error will be documented. "Oh, I don't have to document it," the tester says. "It's not in my component."
======================================================
6. Fanny watched one tester spend the better part of several hours testing the scroll bars on a web-based enterprise system. The scroll bars were, of course, part of web browser, not the system being tested.
======================================================
7. Jerry asked a development manager if the developers on her project unit tested their code. "Absolutely," she said. "We unit test almost all the code."
"Almost?" Jerry asked. "Which code don't you unit test?"
"Oh, some of the code is late, so of course we don't have time to unit test that or we'd have to delay the start of systems test."
======================================================
8. One of Christine's clients conducted an all-day Saturday BugFest, where developers were rewarded with cash for finding bugs in their latest release candidate. They found 282 bugs, which convinced them they were “close to shipping.”
They were so happy with the result that they did it again. This time, they found 343 new bugs - which convinced them they were “on the verge” of shipping.
======================================================
9. A general manager was on the carpet because a recently shipped product was proving terribly buggy in the hands of customers. Jerry asked him why he allowed the product to ship, and he said, "Because our tests proved that it was correct."
======================================================
10. Another manager claimed to Noreen that he knew that their product was ready to ship because "we've run 600,000 test cases and nothing crashed the system."
======================================================
11. When Jerry asked about performance testing, one of his clients said, "We've already done that."
"Really?" said Jerry. "What exactly have you done?"
"We ran the system with one user, and the response time was about ten milliseconds. Then we ran it with two users and the response time was twenty milliseconds. With three users, it was thirty milliseconds."
"Interesting, Jerry responded. "But the system is supposed to support at least a hundred simultaneous users. So what response time did you get when it was fully loaded?"
"Oh, that test would have been too hard to set up, and anyway, it's not necessary. Obviously, the response time would be one second - ten milliseconds per user times one-hundred users."
======================================================
12. Jerry's client calls an emergency meeting to find out "why testing is holding up our product shipment." In the meeting, the testers present 15 failed tests that show that the product did not even build correctly, so it couldn't be tested. They discuss each of the 15 problems with the developers, after which the development lead writes and email summary of the meeting reporting that there are only two "significant" problems.
The email goes to the development manager, the marketing manager, and all the developers who attended the meeting. None of the testers present are included in the cc-list, so none of them even know that the email was sent.
======================================================
13. Johnson watches a tester uncover five bugs in a component, but instead of logging the bugs in the bug database, the tester goes directly to the developer of the component and reports them orally. Johnson asks why the tester didn't record the bugs, and he replies, "If I do that, she (the developer) screams at me because it makes her look bad."
======================================================
14. Tim reviews a client's test plan and notices that there is no plan to test one of the components. When he asks the test manager (who is new to the company) why it's missing, he's told, "We don't need to worry about that. The development manager assures me that this developer is very careful and conscientious."
======================================================
So, were you able to extract meta-information from these ten examples? What did each of them tell you (or hint to you) about the quality of the information from testing - the accuracy, relevance, clarity, and comprehensiveness? Remember, though, that these are merely hints. Perhaps the Sheltie pooped on the rug because he had some medical problem that would need a veterinarian's help.
Perhaps there's a non-obvious explanation behind each of these Sheltie Situations, too, so you always have to follow-up on the hints they provide to validate your intuition or find another interpretation. And, even if your intuition is right on target, you probably won't have an easy time convincing others that you're correct, so you will have to gather other evidence, or other allies, to influence the people who can influence the situation.
When Dani asked the Sheltie's owner why she thought the pup was whining at the front door, the woman said, "I think I've spoiled him. He just wants to go out and play all the time, but I have too much housework to do - especially since I spend so much time cleaning up the mess on the rug."
When a problem persists in spite of how obvious the solution is to you, you aren't going to be able to convince others to solve the problem until you find out how they are rationalizing away the evidence that's so apparent to you. So we need to know how people immunize themselves against information, and what we can do about it.
(Adapted from Perfect Software: and Other Illusions about Testing)
http://www.smashwords.com/books/view/25400
Because of their psychological distance from the problems, external consultants are often able to see information that escapes the eyes and ears of their clients. Dani (Jerry's wife) tells this story about one of her consulting triumphs in the dog world:
======================================================
A woman with a Sheltie puppy was at her wits' end because "he always poops on the living room rug." She loved the little guy, so before giving up and taking him to the Humane Society, she came to Dani for a consultation. Dani listened to the woman describe the problem, then asked, "Are there any other behavior problems you've noticed?"
The woman thought about that for a while, then said, "Well, yes, there is one. He has this annoying habit of scratching on the front door and whining."
======================================================
It's easy to laugh at this woman's inability to see the connection between the two "problems," but that sort of blindness is typical of people who are too close, too emotionally involved, with a situation. Learning to recognize such free information is one of the secrets of successful testing. Using it, you can learn quickly about the quality of an organization's products or the quality of their information obtained from machine testing.
Here are some "Sheltie stories" from our consulting practices. Test yourself by seeing what information you can derive from them about the quality of a product or the quality of the information that's been obtained about the product through testing:
======================================================
1. Jerry is asked to help an organization assess its development processes, including testing. Jerry asks the test manager, "Do you use specs to test against?"
Test manager replies, "Yes, we certainly do."
Jerry: "May I see them?"
Test manager: "Sure, but we don’t know where they are."
======================================================
2. Jerry is called in to help a product development organization's testing group. He learns that they are testing a product that has about 40,000 lines of code.
"The problem," says the test manager, "is with our bug database."
"What's wrong with it," Jerry asks. "Is it buggy?"
"No, it's very reliable, but once it holds more than about 14,000 bug reports, its performance starts to degrade very rapidly. We need you to show us how to improve the performance so we can handle more bug reports."
======================================================
3. Bunny is asked to help improve the testing of a large product that has 22 components. The client identifies "the worst three components" by the high number of bugs found per line of code. Bunny asks which are the best components and is given the very low bugs per line of code figures for each.
She then examines the configuration management logs and discovers that for each of these three "best" components, more than 70 per cent of their code has not yet successfully been checked in for a build.
======================================================
4. Trudy is invited to help a development manager evaluate his testing department's work. She starts by looking at the reports he receives on the number and severity of bugs found each week. The reports are signed by the test manager, but Trudy notices that the severity counts have been whited out and written over.
"Those are corrections by the product development manager," the development manager explains. "Sometimes the testers don't assign the proper severity, so the development manager corrects them."
Using her fingernail, Trudy scratches off the whiteout. Under each highest severity count is a higher printed number. Under each lowest severity count is a lower printed number.
======================================================
5. Jerry is watching a tester running tests on one component of a software product. As the tester is navigating to the target component, Jerry notices an error in one of the menus. The tester curses and navigates around the error by using a direct branch.
Jerry compliments the tester on his ingenuity and resourcefulness, then asks how the error will be documented. "Oh, I don't have to document it," the tester says. "It's not in my component."
======================================================
6. Fanny watched one tester spend the better part of several hours testing the scroll bars on a web-based enterprise system. The scroll bars were, of course, part of web browser, not the system being tested.
======================================================
7. Jerry asked a development manager if the developers on her project unit tested their code. "Absolutely," she said. "We unit test almost all the code."
"Almost?" Jerry asked. "Which code don't you unit test?"
"Oh, some of the code is late, so of course we don't have time to unit test that or we'd have to delay the start of systems test."
======================================================
8. One of Christine's clients conducted an all-day Saturday BugFest, where developers were rewarded with cash for finding bugs in their latest release candidate. They found 282 bugs, which convinced them they were “close to shipping.”
They were so happy with the result that they did it again. This time, they found 343 new bugs - which convinced them they were “on the verge” of shipping.
======================================================
9. A general manager was on the carpet because a recently shipped product was proving terribly buggy in the hands of customers. Jerry asked him why he allowed the product to ship, and he said, "Because our tests proved that it was correct."
======================================================
10. Another manager claimed to Noreen that he knew that their product was ready to ship because "we've run 600,000 test cases and nothing crashed the system."
======================================================
11. When Jerry asked about performance testing, one of his clients said, "We've already done that."
"Really?" said Jerry. "What exactly have you done?"
"We ran the system with one user, and the response time was about ten milliseconds. Then we ran it with two users and the response time was twenty milliseconds. With three users, it was thirty milliseconds."
"Interesting, Jerry responded. "But the system is supposed to support at least a hundred simultaneous users. So what response time did you get when it was fully loaded?"
"Oh, that test would have been too hard to set up, and anyway, it's not necessary. Obviously, the response time would be one second - ten milliseconds per user times one-hundred users."
======================================================
12. Jerry's client calls an emergency meeting to find out "why testing is holding up our product shipment." In the meeting, the testers present 15 failed tests that show that the product did not even build correctly, so it couldn't be tested. They discuss each of the 15 problems with the developers, after which the development lead writes and email summary of the meeting reporting that there are only two "significant" problems.
The email goes to the development manager, the marketing manager, and all the developers who attended the meeting. None of the testers present are included in the cc-list, so none of them even know that the email was sent.
======================================================
13. Johnson watches a tester uncover five bugs in a component, but instead of logging the bugs in the bug database, the tester goes directly to the developer of the component and reports them orally. Johnson asks why the tester didn't record the bugs, and he replies, "If I do that, she (the developer) screams at me because it makes her look bad."
======================================================
14. Tim reviews a client's test plan and notices that there is no plan to test one of the components. When he asks the test manager (who is new to the company) why it's missing, he's told, "We don't need to worry about that. The development manager assures me that this developer is very careful and conscientious."
======================================================
So, were you able to extract meta-information from these ten examples? What did each of them tell you (or hint to you) about the quality of the information from testing - the accuracy, relevance, clarity, and comprehensiveness? Remember, though, that these are merely hints. Perhaps the Sheltie pooped on the rug because he had some medical problem that would need a veterinarian's help.
Perhaps there's a non-obvious explanation behind each of these Sheltie Situations, too, so you always have to follow-up on the hints they provide to validate your intuition or find another interpretation. And, even if your intuition is right on target, you probably won't have an easy time convincing others that you're correct, so you will have to gather other evidence, or other allies, to influence the people who can influence the situation.
When Dani asked the Sheltie's owner why she thought the pup was whining at the front door, the woman said, "I think I've spoiled him. He just wants to go out and play all the time, but I have too much housework to do - especially since I spend so much time cleaning up the mess on the rug."
When a problem persists in spite of how obvious the solution is to you, you aren't going to be able to convince others to solve the problem until you find out how they are rationalizing away the evidence that's so apparent to you. So we need to know how people immunize themselves against information, and what we can do about it.
(Adapted from Perfect Software: and Other Illusions about Testing)http://www.smashwords.com/books/view/25400