Blogger Jon Jagger describes himself as a self employed software consultant-mentor-trainer-programmer etc specializing in agile software development (people and process), test driven development, deliberate practice, design, analysis, OO, UML, curly bracket languages (C#, C++, Java)
Jon recently published a list of 10 (+1) books he would like to have if marooned on a desert island. It's a fascinating list, and not only because it contains three of my books. You can read Jon's justification for each book at:
http://jonjagger.blogspot.com/2009/09/desert-island-books.html
But here they are. How many have you read?
* [1] Kevin Ashurst. (1977 Long out of print). World Class Match Fishing, Cassell, ISBN 0304-297291.
* [2] Phillip Pullman. (1995). The Northern Lights (The Golden Compass in USA, Knopf), Scholastic, ISBN 043995178X
* [3] Douglas Adams. (1979). The Hitch Hikers Guide to the Galaxy, Pan, ISBN 0330258648
* [4] Gerald Weinberg. (1985). The Secrets of Consulting, Dorset House, ISBN 0932633013
* [5] Gerald Weinberg. (1998). The Psychology of Computer Programming: Silver Anniversary Edition, Dorset House, ISBN 0932633420
* [6] Monty Python. (2001). The Life of Brian (screenplay), Metheun, ISBN 0413741303
* [7] Jon Bentley, (1989). Programming Pearls, Addison Wesley, ISBN 0201103311
* [8] Fred Brooks (1985 2nd edition). The Mythical Man Month, Addison Wesley, ISBN 0201835959
* [9] Peter Senge (2006 2nd edition). The Fifth Discipline, Random House, ISBN 1905211201
* [10] Gerald Weinberg (2001). Introduction to General Systems Thinking Silver anniversary edition, Dorset House, ISBN 0932633498
* [11] John Gall (2002 3rd edition). The Systems Bible: The Beginner's Guide to Systems Large and Small , General Systemantics Press, ISBN 0961825170
So, that's Jon's list. What would be on yours? Are any of Jon's choices books that you wouldn't care to have on your desert island?
p.s. Jon's going to be at AYE Conference in November, and so will I, in case you would like to discuss choosing books and other topics with us.
Saturday, September 19, 2009
Thursday, September 10, 2009
50 more ways to improve your business
Wolfram Arnold writes:
Here are the points I've transcribed from the meeting today. At some point I lost count as to which one was odd and which one was even and I just recorded them all (touch-typing helped :-):
---------------------
Jerry Weinberg 8/20, 50 things to improve business
2. back up everything
4. Rule: do nothing, revised with 3 caveats: a) don't do it if there's someone that can do it better; b) don't do it if there's someone that can do it adequately; c) if it makes me really happy, do it anyway; d) if someone can do it 85% as well as I do, let them do it; e) anything not worth doing is not worth doing right; f) if in doubt charge for sales trips
6. make them pay something, with their time, their money -- if they don't pay for it they don't value it
8. if it's not on paper, don't do it
9. listen to what other people are telling you
10. don't communicate to somebody, but communicate with somebody
11. always have an exit strategy
12. make them feel like your client has a part in the final outcome; make sure they have their fingerprints on it
13. listen for what they're not saying
14. listen to the "music", body language, intonation
15. always be ready to sell your product
16. if you find yourself reluctant to sell your product, there's something wrong with it
17. any successful services company has some fixed priced product to sell
18. given them entry points that they can buy
19. recurring revenue model, e.g. via contract maintenance plans, or follow-through
20. have a follow-through clause in contract so you can know how you're doing
21. charge more money if they don't want you to come back after some time, e.g. 3 months
22. if you just build it they probably won't come
23. manage expectations, book: Managing Expectations, by Naomi Karten
24. Time spent in reconnaissance is time well spent
25. You can observe a whole lot just by watching (Yogi Berra)
26. Go hard or go home; fully commit all resources needed, or kill it mercilessly
27. Commit enough to learn what you have to learn to find out whether it's worth pursuing or killing
28. People who work in an Agile/iterative way often fail to do the discovery
29. Ideas by themselves aren't as valuable as you think they are; don't guard them too closely
30. Nothing is as dangerous as an idea, esp. if it's the only idea you have
31. Never rest on your past successes; there is always something more you could be doing; if you're not learning, you're dead
32. Sharing competitive advantages brings 10-fold rewards; give it away, it comes back
33. Being able to say 'no'
34. Research clients as if you were hiring them
35. Recognize that every client is unique.
36. You don't have to remember everything to succeed.
37. It's ok to let a client go if it's not the right fit; you should organize your business such that it's ok to let a client go, i.e. don't be over-dependent on any single client
38. The best way to build a business is to stay in business; stay around, build a reputation and credibility
39. Actively solicit feedback from clients; actively extract the feedback, e.g. watch the audience
40. Don't be alone in your work; have someone to talk to
41. Honor the people who are your sounding board and bring feedback, e.g. life partners, friends, ...
42. Anything that's annoying or repetitive should be automated or stopped
43. Track your budget & cost every month
44. Don't make mistakes over your budget or your cost.
45. Don't spend your money on office decoration, esp. if your clients don't come to your office
46. Always try someone out before you hire them
47. Don't fall for the big lies: "we're just about to get funding" "our data is clean" "your check is in the mail" "we're going to sign it next month, just keep working" "don't worry about the contract" ...
48. Preventing any one of these mistakes will pay for this conference
49. Double your reading speed
50. Choose not to read a lot; don't read stuff that's not worth reading
51. Stay off Facebook & Twitter
52. Sometimes you can save money by spending money; and sometimes the reverse. Learn to tell the difference
- Thanks for the great job, Wolfram
Here are the points I've transcribed from the meeting today. At some point I lost count as to which one was odd and which one was even and I just recorded them all (touch-typing helped :-):
---------------------
Jerry Weinberg 8/20, 50 things to improve business
2. back up everything
4. Rule: do nothing, revised with 3 caveats: a) don't do it if there's someone that can do it better; b) don't do it if there's someone that can do it adequately; c) if it makes me really happy, do it anyway; d) if someone can do it 85% as well as I do, let them do it; e) anything not worth doing is not worth doing right; f) if in doubt charge for sales trips
6. make them pay something, with their time, their money -- if they don't pay for it they don't value it
8. if it's not on paper, don't do it
9. listen to what other people are telling you
10. don't communicate to somebody, but communicate with somebody
11. always have an exit strategy
12. make them feel like your client has a part in the final outcome; make sure they have their fingerprints on it
13. listen for what they're not saying
14. listen to the "music", body language, intonation
15. always be ready to sell your product
16. if you find yourself reluctant to sell your product, there's something wrong with it
17. any successful services company has some fixed priced product to sell
18. given them entry points that they can buy
19. recurring revenue model, e.g. via contract maintenance plans, or follow-through
20. have a follow-through clause in contract so you can know how you're doing
21. charge more money if they don't want you to come back after some time, e.g. 3 months
22. if you just build it they probably won't come
23. manage expectations, book: Managing Expectations, by Naomi Karten
24. Time spent in reconnaissance is time well spent
25. You can observe a whole lot just by watching (Yogi Berra)
26. Go hard or go home; fully commit all resources needed, or kill it mercilessly
27. Commit enough to learn what you have to learn to find out whether it's worth pursuing or killing
28. People who work in an Agile/iterative way often fail to do the discovery
29. Ideas by themselves aren't as valuable as you think they are; don't guard them too closely
30. Nothing is as dangerous as an idea, esp. if it's the only idea you have
31. Never rest on your past successes; there is always something more you could be doing; if you're not learning, you're dead
32. Sharing competitive advantages brings 10-fold rewards; give it away, it comes back
33. Being able to say 'no'
34. Research clients as if you were hiring them
35. Recognize that every client is unique.
36. You don't have to remember everything to succeed.
37. It's ok to let a client go if it's not the right fit; you should organize your business such that it's ok to let a client go, i.e. don't be over-dependent on any single client
38. The best way to build a business is to stay in business; stay around, build a reputation and credibility
39. Actively solicit feedback from clients; actively extract the feedback, e.g. watch the audience
40. Don't be alone in your work; have someone to talk to
41. Honor the people who are your sounding board and bring feedback, e.g. life partners, friends, ...
42. Anything that's annoying or repetitive should be automated or stopped
43. Track your budget & cost every month
44. Don't make mistakes over your budget or your cost.
45. Don't spend your money on office decoration, esp. if your clients don't come to your office
46. Always try someone out before you hire them
47. Don't fall for the big lies: "we're just about to get funding" "our data is clean" "your check is in the mail" "we're going to sign it next month, just keep working" "don't worry about the contract" ...
48. Preventing any one of these mistakes will pay for this conference
49. Double your reading speed
50. Choose not to read a lot; don't read stuff that's not worth reading
51. Stay off Facebook & Twitter
52. Sometimes you can save money by spending money; and sometimes the reverse. Learn to tell the difference
- Thanks for the great job, Wolfram
Sunday, August 23, 2009
50 Ways to Improve Your Business
At last week's BizConf, I ran a session based on an idea from Dwayne Phillips, where a bunch of independent businesspeople brainstormed 50 small ways to improve your business. The ideas flew fast and furious, so I assigned two participants, each to capture alternate ideas. Even so, it was hard to keep up with the flow. The lists were quite overwhelming, so I'll present them in two separate blogs. Today, it will be Jason Seifer's list. Jason said, "I wound up just logging everything, because I touch type like in your first rule."
So, here's Jason's list:
1. Learn to touch type.
2. Back-up everything. Every day.
3. Keep as much stuff online as possible.
4. Do nothing. Don't do something if someone can do it better. Don't do it if someone else can do it adequately. Do it anyway if it makes you happy. Don't do it if someone else can do it 85% as well as you.
5. Anything not worth doing is not worth doing right.
6. If in doubt charge for sales trips. If they're not willing to pay for it they don't value it.
7. If it's not on paper don't do it. If there's money involved it has to be written down. Never agree to money over the phone. If on phone, confirm in writing.
8. Listen to what other people are telling you. Instead of communicating to somebody, communicate with somebody.
9. Always have an exit strategy.
10. Make your client feel like they have a final part in the outcome. Make sure they have their fingerprints in it.
11. Listen to what they're not saying. Listen to body language and tone of voice.
12. Always be ready to sell your product if they're interested. And if they're not.
13. If you're bashful about your product, there's something wrong with your product.
14. Any successful services company has some fixed product that they sell. Have a ladder leading to your ultimate product. Don't make one big leap to the final product.
15. Make sure your contracts have some kind of follow-through so you can see how they actually came out. You won't know if you're doing well if they don't come back.
16. If you just build it they probably won't come. Alternately: if you build it be prepared to wait.
17. Manage expectations.
18. Time spent in recon is time well spent. But you have to be watching. "You can observe a whole lot just by watching."
19. Go hard or go home. Fully commit the resources to make something work or mercilessly kill it. You may not always know when it's time to kill it.
20. Ideas by themselves are not as valuable as you think they are. Ideas aren't worth anything so don't guard them too closely. "There's nothing as dangerous as an idea if it's the only idea you have."
21. Never rest on your past successes, there's always something more you could be doing. If you're not learning you're dead.
22. Sharing competitive advantages brings back advantages ten-fold.
23. Be able to say "No" to a potential client.
24. Research your clients as if you were doing the hiring.
25. If you're in the services business every client is unique. "We're different" "You are, just like everybody else."
26. You don't have to remember everything to succeed.
27. It's always ok to let a client go if it's not the right fit any more.You should organize your business so it's ok to let any one client go at any time.
28. The best way to build a business is to stay in business. Build your business so that you hang around. You may not make a lot of money but you build your reputation.
29. Actively solicit feedback from clients.
30. Actively extract the feedback. There's a lot of feedback you may not pick up on.
31. Make sure you're not alone in your role. This way someone can be honest with you.
32. Anything that's annoying and repetitive should be automated or stopped.
33. Track your budget/cost every month.
34. Don't make sampling mistakes about your budget and cost. One of the major mistakes that people fail at is expecting that income will stay the same. You might land a big client this month but might not have one next month so don't overspend.
35. Don't spend your money on office decorations. Very few of us are in a business where clients come to your office. You don't want your customers to think you're spending their money on their office furniture.
36. Make sure you've worked with someone before you hire them (if possible).
37. Learn the big lies you get from clients.
38. Double your reading speed.
39. Cut down on what you read.
40. Stay off facebook and twitter unless you can relate it to your business.
41. Sometimes you can save money by spending money.
42. Sometimes you can save money by not spending money.
43. Everyone who wants to sell you something will tell you it saves money.
44. The examples might not always teach what they're supposed to.
45. Many minds can do a better job than single minds. But not necessarily so.
46. You have to be organized.
47. Know your own limitations.
48. Learn quickly whether or not you want to work with someone.
49. If you see an organization with no enthusiasm for what they're doing they probably don't have enthusiasm for what you're offering.
50. If an organization doesn't care enough to organize, nothing is going to happen.
51. Don't mistake a solution idea for a problem definition.
52. People want a recipe but no recipe works for every organization.
So, that's Jason's (half) list. Each one could probably be a blog entry, at least, so if you want clarification, or to clarify, add a comment. I'll post Wolfram's (half) list when the traffic on this one dies down.
So, here's Jason's list:
1. Learn to touch type.
2. Back-up everything. Every day.
3. Keep as much stuff online as possible.
4. Do nothing. Don't do something if someone can do it better. Don't do it if someone else can do it adequately. Do it anyway if it makes you happy. Don't do it if someone else can do it 85% as well as you.
5. Anything not worth doing is not worth doing right.
6. If in doubt charge for sales trips. If they're not willing to pay for it they don't value it.
7. If it's not on paper don't do it. If there's money involved it has to be written down. Never agree to money over the phone. If on phone, confirm in writing.
8. Listen to what other people are telling you. Instead of communicating to somebody, communicate with somebody.
9. Always have an exit strategy.
10. Make your client feel like they have a final part in the outcome. Make sure they have their fingerprints in it.
11. Listen to what they're not saying. Listen to body language and tone of voice.
12. Always be ready to sell your product if they're interested. And if they're not.
13. If you're bashful about your product, there's something wrong with your product.
14. Any successful services company has some fixed product that they sell. Have a ladder leading to your ultimate product. Don't make one big leap to the final product.
15. Make sure your contracts have some kind of follow-through so you can see how they actually came out. You won't know if you're doing well if they don't come back.
16. If you just build it they probably won't come. Alternately: if you build it be prepared to wait.
17. Manage expectations.
18. Time spent in recon is time well spent. But you have to be watching. "You can observe a whole lot just by watching."
19. Go hard or go home. Fully commit the resources to make something work or mercilessly kill it. You may not always know when it's time to kill it.
20. Ideas by themselves are not as valuable as you think they are. Ideas aren't worth anything so don't guard them too closely. "There's nothing as dangerous as an idea if it's the only idea you have."
21. Never rest on your past successes, there's always something more you could be doing. If you're not learning you're dead.
22. Sharing competitive advantages brings back advantages ten-fold.
23. Be able to say "No" to a potential client.
24. Research your clients as if you were doing the hiring.
25. If you're in the services business every client is unique. "We're different" "You are, just like everybody else."
26. You don't have to remember everything to succeed.
27. It's always ok to let a client go if it's not the right fit any more.You should organize your business so it's ok to let any one client go at any time.
28. The best way to build a business is to stay in business. Build your business so that you hang around. You may not make a lot of money but you build your reputation.
29. Actively solicit feedback from clients.
30. Actively extract the feedback. There's a lot of feedback you may not pick up on.
31. Make sure you're not alone in your role. This way someone can be honest with you.
32. Anything that's annoying and repetitive should be automated or stopped.
33. Track your budget/cost every month.
34. Don't make sampling mistakes about your budget and cost. One of the major mistakes that people fail at is expecting that income will stay the same. You might land a big client this month but might not have one next month so don't overspend.
35. Don't spend your money on office decorations. Very few of us are in a business where clients come to your office. You don't want your customers to think you're spending their money on their office furniture.
36. Make sure you've worked with someone before you hire them (if possible).
37. Learn the big lies you get from clients.
38. Double your reading speed.
39. Cut down on what you read.
40. Stay off facebook and twitter unless you can relate it to your business.
41. Sometimes you can save money by spending money.
42. Sometimes you can save money by not spending money.
43. Everyone who wants to sell you something will tell you it saves money.
44. The examples might not always teach what they're supposed to.
45. Many minds can do a better job than single minds. But not necessarily so.
46. You have to be organized.
47. Know your own limitations.
48. Learn quickly whether or not you want to work with someone.
49. If you see an organization with no enthusiasm for what they're doing they probably don't have enthusiasm for what you're offering.
50. If an organization doesn't care enough to organize, nothing is going to happen.
51. Don't mistake a solution idea for a problem definition.
52. People want a recipe but no recipe works for every organization.
So, that's Jason's (half) list. Each one could probably be a blog entry, at least, so if you want clarification, or to clarify, add a comment. I'll post Wolfram's (half) list when the traffic on this one dies down.
Tuesday, August 04, 2009
The Evolution of an Exercise
(Rhonda asks Jerry for a little consultation.)
RHONDA: I'm a little nervous about my "speech" at a conference in a couple of weeks and wanted to see if you have time for some feedback.
JERRY: First feedback: If you weren't nervous, then you'd give a boring speech, guaranteed. Breathe into your nervousness and it becomes excitement.
RHONDA: It's the first time I'm presenting representing my company, the virginal gig so to speak, and I'm unsure what to prepare to make best use of the limited time while not knowing how many participants are going to join my session.
JERRY: Don't worry about "using time." Design the session so that the most important things come first (or early). That way if you "run out of time," you've covered as much as you could have.
RHONDA: "Whatever happens, happens" has been my mantra for a while already, and most of the "what-if's" won't get my blood pressure up (at least not until the hour before I go up front). To be honest, I've thought about this session for so many weeks now and feel so close to it that I feel stuck, like I have blinders on, and can't see alternative options how else to execute it.
JERRY: Stop thinking about it.
Instead of thinking, start doing. Figure out a way to practice it on some friends. Invite some of those friends over for wine and cheese or beer and pretzels or something, then use them as a surrogate audience and listen to their feedback.
RHONDA: I'm advertised thus: Starting Your Own Business - Building the Life You Want. The purpose of this presentation is to share with the audience the journey the presenter followed from international student via international employee and trailing spouse to expat coach and owner of her own company.
JERRY: Rewrite this, if only for your own use. The purpose is not to "share experiences with them." That's a means of achieving the purpose, which is something like "helping the audience members to succeed in starting their own business." IOW, more about them; less about you.
RHONDA: Rhonda draws on over 10 years of personal expatriate experience that made her want to support others. The audience will hear tips and descriptions of how and where she got the necessary information to dot the i's and cross the t's en route to realizing her dream. She will also make time for and encourage the audience to share their experiences and brainstorm ideas to make sure everybody who wants to start a business will leave her presentation motivated and informed.
Objectives I have for the "speech" (assuming participants come to hear about how to start their own businesses - is that a mistake?
JERRY: It will reduce your audience, which could be good or bad. Who else would benefit from this session?
RHONDA: Should I assume anything at all?):
a) clarify their vision / focus their goals
b) raise awareness of hidden obstacles
c) identify concrete action steps to get started
Writing a business plan answers all three objectives.
(I've compiled a handbook with information about expatriate work permits, business structure comparison, business owner character traits, and useful links and resources to cover more start-up info. The handbook will be available either as print-out or .pdf file in exchange for their email address after the presentation.)
JERRY: Emphasize the handbook. People like takeaways.
Also emphasize what Eisenhower said: "The plan is nothing; the planning is everything." Maybe you could have them step through your planning process with you, each one (or team) doing their own planning steps as you go along.
RHONDA: Here are some of the parameters:
75 minutes
Should expect between 20 and 50 participants
Don't know exact number
Can't put the whole start-up process in 75 minutes
JERRY: So do selected parts, most important first.
RHONDA: I won't spend 75 minutes lecturing
JERRY: You'd better not.
RHONDA: Session Outline
(5-10 minutes intro/warm-up)
Exploration: 10-15 minutes to explain benefits, structure, and reasoning for a business plan
Introduce business plan segments (e.g. client and product profile, market profile, marketing strategy, organizational structure and finances)
Set up exercise: If I have 20 participants, I'll use one new venture/market scenario. One group per segment. Each group reads background information I provide (or would it be easier if they make up their own venture and background?) and answer business plan questions. Time: ca. 5 minutes
JERRY: Definitely better if they use their own, and you do it incrementally. You don't need to do the overview up front. You want to get them doing things more quickly than that. As it is, you have more than 1/2 hour before they do anything.
For example, pick the part of planning that's most important and start with that. When you've done that, and everyone has done that and questions are answered, move to the next most important. Do as many as you can cover properly without rushing. Then, when you see that ten minutes are left, conclude with an overview of all the segments they need to do to have a plan, and tell them about the handbook--again.
RHONDA: If I have 50 participants, I'll use two or three new venture/market scenarios, e.g. dog-wash salon in Seattle, pizzeria in Paris, recruitment office in Barcelona?
JERRY: No, too much time explaining the scenarios, which do them no good. Doing their own scenarios saves this time and ensures real interest in the exercises. If someone doesn't have one of their own, have them pair with someone who has one of their own.
RHONDA: Discovery/Application:
Participants write a business plan for their venture. Time: ca. 20 minutes.
Debrief/Application: In whole group, write executive summary for each venture, taking most important bits from each segment on flip chart, (take a picture, send it to them afterward with a thank-you note). Time: ca. 30 minutes.
JERRY: You can do this with lessons they learned from each different
startup, which lets them see what different startups have in common, and
what are the exceptions.
RHONDA: Question: Is 30 minutes enough debrief-time for an exercise like this and group size of 50 people?
JERRY: No. At least five days would be required to do it properly. But, you don't have that, so do what you can. If you do this incrementally, you can extract lessons after each segment, then do as many segments as
develop naturally.
RHONDA: Am I trying to cram too much in in general?
JERRY: Yes.
RHONDA: Ideas for a possible shorter exercise that would make 50 people feel involved and stimulated?
JERRY: Basically the same exercise, but chopped up and presented incrementally.
JERRY: BTW, if you really have 50, best to have them work in teams of 3-5 people, each formed around one person who has a specific startup in mind. To do this, you have individuals write signs that say, "Dog- walking business," or "Real-estate for the rich and famous," or "Coffin upholsterer," or whatever they're actually thinking of starting. Those that have signs hold them up, and people attach themselves to the ones they're interested in to make teams.
Does this help?
RHONDA: Yes!
RHONDA: I'm a little nervous about my "speech" at a conference in a couple of weeks and wanted to see if you have time for some feedback.
JERRY: First feedback: If you weren't nervous, then you'd give a boring speech, guaranteed. Breathe into your nervousness and it becomes excitement.
RHONDA: It's the first time I'm presenting representing my company, the virginal gig so to speak, and I'm unsure what to prepare to make best use of the limited time while not knowing how many participants are going to join my session.
JERRY: Don't worry about "using time." Design the session so that the most important things come first (or early). That way if you "run out of time," you've covered as much as you could have.
RHONDA: "Whatever happens, happens" has been my mantra for a while already, and most of the "what-if's" won't get my blood pressure up (at least not until the hour before I go up front). To be honest, I've thought about this session for so many weeks now and feel so close to it that I feel stuck, like I have blinders on, and can't see alternative options how else to execute it.
JERRY: Stop thinking about it.
Instead of thinking, start doing. Figure out a way to practice it on some friends. Invite some of those friends over for wine and cheese or beer and pretzels or something, then use them as a surrogate audience and listen to their feedback.
RHONDA: I'm advertised thus: Starting Your Own Business - Building the Life You Want. The purpose of this presentation is to share with the audience the journey the presenter followed from international student via international employee and trailing spouse to expat coach and owner of her own company.
JERRY: Rewrite this, if only for your own use. The purpose is not to "share experiences with them." That's a means of achieving the purpose, which is something like "helping the audience members to succeed in starting their own business." IOW, more about them; less about you.
RHONDA: Rhonda draws on over 10 years of personal expatriate experience that made her want to support others. The audience will hear tips and descriptions of how and where she got the necessary information to dot the i's and cross the t's en route to realizing her dream. She will also make time for and encourage the audience to share their experiences and brainstorm ideas to make sure everybody who wants to start a business will leave her presentation motivated and informed.
Objectives I have for the "speech" (assuming participants come to hear about how to start their own businesses - is that a mistake?
JERRY: It will reduce your audience, which could be good or bad. Who else would benefit from this session?
RHONDA: Should I assume anything at all?):
a) clarify their vision / focus their goals
b) raise awareness of hidden obstacles
c) identify concrete action steps to get started
Writing a business plan answers all three objectives.
(I've compiled a handbook with information about expatriate work permits, business structure comparison, business owner character traits, and useful links and resources to cover more start-up info. The handbook will be available either as print-out or .pdf file in exchange for their email address after the presentation.)
JERRY: Emphasize the handbook. People like takeaways.
Also emphasize what Eisenhower said: "The plan is nothing; the planning is everything." Maybe you could have them step through your planning process with you, each one (or team) doing their own planning steps as you go along.
RHONDA: Here are some of the parameters:
75 minutes
Should expect between 20 and 50 participants
Don't know exact number
Can't put the whole start-up process in 75 minutes
JERRY: So do selected parts, most important first.
RHONDA: I won't spend 75 minutes lecturing
JERRY: You'd better not.
RHONDA: Session Outline
(5-10 minutes intro/warm-up)
Exploration: 10-15 minutes to explain benefits, structure, and reasoning for a business plan
Introduce business plan segments (e.g. client and product profile, market profile, marketing strategy, organizational structure and finances)
Set up exercise: If I have 20 participants, I'll use one new venture/market scenario. One group per segment. Each group reads background information I provide (or would it be easier if they make up their own venture and background?) and answer business plan questions. Time: ca. 5 minutes
JERRY: Definitely better if they use their own, and you do it incrementally. You don't need to do the overview up front. You want to get them doing things more quickly than that. As it is, you have more than 1/2 hour before they do anything.
For example, pick the part of planning that's most important and start with that. When you've done that, and everyone has done that and questions are answered, move to the next most important. Do as many as you can cover properly without rushing. Then, when you see that ten minutes are left, conclude with an overview of all the segments they need to do to have a plan, and tell them about the handbook--again.
RHONDA: If I have 50 participants, I'll use two or three new venture/market scenarios, e.g. dog-wash salon in Seattle, pizzeria in Paris, recruitment office in Barcelona?
JERRY: No, too much time explaining the scenarios, which do them no good. Doing their own scenarios saves this time and ensures real interest in the exercises. If someone doesn't have one of their own, have them pair with someone who has one of their own.
RHONDA: Discovery/Application:
Participants write a business plan for their venture. Time: ca. 20 minutes.
Debrief/Application: In whole group, write executive summary for each venture, taking most important bits from each segment on flip chart, (take a picture, send it to them afterward with a thank-you note). Time: ca. 30 minutes.
JERRY: You can do this with lessons they learned from each different
startup, which lets them see what different startups have in common, and
what are the exceptions.
RHONDA: Question: Is 30 minutes enough debrief-time for an exercise like this and group size of 50 people?
JERRY: No. At least five days would be required to do it properly. But, you don't have that, so do what you can. If you do this incrementally, you can extract lessons after each segment, then do as many segments as
develop naturally.
RHONDA: Am I trying to cram too much in in general?
JERRY: Yes.
RHONDA: Ideas for a possible shorter exercise that would make 50 people feel involved and stimulated?
JERRY: Basically the same exercise, but chopped up and presented incrementally.
JERRY: BTW, if you really have 50, best to have them work in teams of 3-5 people, each formed around one person who has a specific startup in mind. To do this, you have individuals write signs that say, "Dog- walking business," or "Real-estate for the rich and famous," or "Coffin upholsterer," or whatever they're actually thinking of starting. Those that have signs hold them up, and people attach themselves to the ones they're interested in to make teams.
Does this help?
RHONDA: Yes!
Saturday, July 25, 2009
How to Be Happy, Though in a Non-supportive Environment
Jerry: Here's another email consulting dialogue that Larry found helpful.
Larry: My name is Larry. I am a software test manager at an insurance company east of Chicago. I have faced challenges when trying to do what I consider good testing. These challenges include standards that mandate scripted test cases, fellow managers who don't want to discuss work, and security policies that don't allow us to quickly get tools that will aid in our testing.
A while back I discovered Cem Kaner's web page and it opened me up to a whole new world of software testing. I learned of people like you, James Bach, Michael Bolton, and several other great thinkers. I wonder how many people spend a career in software development/testing/ management without ever learning of these great people.
Jerry: Quite a few. To have a great career in any profession, you have to reach out and find sources such as these. You need to participate in a conference once in a while (the AYE Conference in November would be my number one choice in your situation). After that, our Problem Solving Leadership workshop would be an ideal goal to aspire to. You should certainly join your local interest groups, and participate.
Larry: This has led me to a question that I hope you can help me with. How smart does someone have to be, to be happier when they read your books?
Jerry: Sometimes, just reading is sufficient, but most of the time, the reader has to begin doing something they weren't doing before. Like the suggestions above. Or like tackling one of the problems you cited--the standards, your fellow managers, or security policies. Or some smaller problem that nags at you.
But only ONE at a time, to learn what works for you and your organization. And what doesn't work.
(If nothing works, you want to be looking for a better place to work.)
Larry: It's almost like you know me. I think I have an NT temperament (QSM Volume 3 helped here) and based on some of the other research I've done it is definitely true that when I have a lot of tasks and become stressed I almost lock up. I rationalize that it is because I can't devote enough time to do a good job on any one thing, but I know I need to just suck it up and handle on item really well. That may go a long way to improving my happiness.
Jerry: From the little I know, I think if it were me, I'd try to find (at least) one of my fellow managers who is willing to spend some time with me discussing things that we could accomplish to improve matters for our company.
But any little thing you could move forward would be educational--even if it "fails" you can extract some learning from the attempt.
Larry: I have read five of your books so far:
* Are Your Lights On?
* Quality Software Management: Volume 1
* Quality Software Management: Volume 3
* Exploring Requirements
* Weinberg on Writing
To be honest I probably can't say "read" in the same sense that you might. I read some areas in depth and browsed others. I'm making second passes through several.
Jerry: That's exactly the way I read.
Larry: My worry is that I feel less happy after having read your books. They have shown me how much I have to learn, and I believe that I'm not in the right environment to continue this learning.
Jerry: At least you haven't run out of things to learn. Now THAT would be really depressing.
As for the environment for learning, no environment can stop you from learning, if you really care. But, yes, you may eventually decide to keep an eye out for an opportunity in a different environment.
Larry: Each step I take towards more critical thinking, a strong thirst for knowledge, a greater understanding of how much I really don't know is a step I'm taking away from my peers at work.
Jerry: As for your peers, by taking a step ahead of them, you may well be modeling a new way for them to be happier. It's called being a "leader."
Larry: This has led to a lot of work related stress and unhappiness, and I'm more unhappy than I have been in a long time.
Jerry: In volume 4 of Quality Software Management, you'll learn about the Satir Change Model, and why significant change is usually preceded by a period of chaos, which might feel unhappy until you realize that it's a natural step on the way to happy change.
Larry: That's interesting. My personal experience says that I do tend to have phases of unhappiness, but I come out of it much stronger. It is just hard to know where I am in my journey at a specific point in time. Am I climbing up or falling down?
Now, I'm not writing to blame your great books for my unhappiness. I just have a feeling that other people have had similar journeys, and I was wondering if you've encountered a similar phenomenon. Do you have any more insights for a young unhappy software tester?
Jerry: I suspect my newest book, Perfect Software might be a good read for your manager and your fellow managers
Larry: I actually forgot about that book on my list. It is sitting on my desk pointing anyone who walks in. I'm hoping someone will pick it up and be interested, but that hasn't happened yet.
Jerry: You need to be more patient, and more aggressive, at the same time. Not easy!
Larry: My name is Larry. I am a software test manager at an insurance company east of Chicago. I have faced challenges when trying to do what I consider good testing. These challenges include standards that mandate scripted test cases, fellow managers who don't want to discuss work, and security policies that don't allow us to quickly get tools that will aid in our testing.
A while back I discovered Cem Kaner's web page and it opened me up to a whole new world of software testing. I learned of people like you, James Bach, Michael Bolton, and several other great thinkers. I wonder how many people spend a career in software development/testing/ management without ever learning of these great people.
Jerry: Quite a few. To have a great career in any profession, you have to reach out and find sources such as these. You need to participate in a conference once in a while (the AYE Conference in November would be my number one choice in your situation). After that, our Problem Solving Leadership workshop would be an ideal goal to aspire to. You should certainly join your local interest groups, and participate.
Larry: This has led me to a question that I hope you can help me with. How smart does someone have to be, to be happier when they read your books?
Jerry: Sometimes, just reading is sufficient, but most of the time, the reader has to begin doing something they weren't doing before. Like the suggestions above. Or like tackling one of the problems you cited--the standards, your fellow managers, or security policies. Or some smaller problem that nags at you.
But only ONE at a time, to learn what works for you and your organization. And what doesn't work.
(If nothing works, you want to be looking for a better place to work.)
Larry: It's almost like you know me. I think I have an NT temperament (QSM Volume 3 helped here) and based on some of the other research I've done it is definitely true that when I have a lot of tasks and become stressed I almost lock up. I rationalize that it is because I can't devote enough time to do a good job on any one thing, but I know I need to just suck it up and handle on item really well. That may go a long way to improving my happiness.
Jerry: From the little I know, I think if it were me, I'd try to find (at least) one of my fellow managers who is willing to spend some time with me discussing things that we could accomplish to improve matters for our company.
But any little thing you could move forward would be educational--even if it "fails" you can extract some learning from the attempt.
Larry: I have read five of your books so far:
* Are Your Lights On?
* Quality Software Management: Volume 1
* Quality Software Management: Volume 3
* Exploring Requirements
* Weinberg on Writing
To be honest I probably can't say "read" in the same sense that you might. I read some areas in depth and browsed others. I'm making second passes through several.
Jerry: That's exactly the way I read.
Larry: My worry is that I feel less happy after having read your books. They have shown me how much I have to learn, and I believe that I'm not in the right environment to continue this learning.
Jerry: At least you haven't run out of things to learn. Now THAT would be really depressing.
As for the environment for learning, no environment can stop you from learning, if you really care. But, yes, you may eventually decide to keep an eye out for an opportunity in a different environment.
Larry: Each step I take towards more critical thinking, a strong thirst for knowledge, a greater understanding of how much I really don't know is a step I'm taking away from my peers at work.
Jerry: As for your peers, by taking a step ahead of them, you may well be modeling a new way for them to be happier. It's called being a "leader."
Larry: This has led to a lot of work related stress and unhappiness, and I'm more unhappy than I have been in a long time.
Jerry: In volume 4 of Quality Software Management, you'll learn about the Satir Change Model, and why significant change is usually preceded by a period of chaos, which might feel unhappy until you realize that it's a natural step on the way to happy change.
Larry: That's interesting. My personal experience says that I do tend to have phases of unhappiness, but I come out of it much stronger. It is just hard to know where I am in my journey at a specific point in time. Am I climbing up or falling down?
Now, I'm not writing to blame your great books for my unhappiness. I just have a feeling that other people have had similar journeys, and I was wondering if you've encountered a similar phenomenon. Do you have any more insights for a young unhappy software tester?
Jerry: I suspect my newest book, Perfect Software might be a good read for your manager and your fellow managers
Larry: I actually forgot about that book on my list. It is sitting on my desk pointing anyone who walks in. I'm hoping someone will pick it up and be interested, but that hasn't happened yet.
Jerry: You need to be more patient, and more aggressive, at the same time. Not easy!
Tuesday, July 07, 2009
Peter Principle Simulated
Some Italian professors have tried to simulate the famous Peter Principle:
"All new members in a hierarchical organization climb the hierarchy until they reach their level of maximum incompetence."
http://www.technologyreview.com/blog/arxiv/23800/
The Peter effect arises from the practice of promoting the best performer at Level N to a position at level N+1.
The Italians also simulated two other promotion policies:
1. Alternately promote first the most competent and then the least competent individuals.
2. Promote individuals at random.
According to their simulations, each of these methods improves the efficiency of an organization over the Peter method.
My thought: They could try promoting on the basis of who is most suited for the next level job. Duh!
Or maybe they could try not mixing the concept of "promotion" with that of "reward."
Or maybe even getting rid of the hierarchical notion altogether.
"People become progressively less competent for jobs they once were well equipped to handle."
Paul proposed his law in 1970, the year after Peters proposed his. Paul claimed his principle was more relevant in high-tech fields, when the complexity of jobs grows faster than the people doing them. The Paul Principle has been virtually forgotten, but I think it is still worth some careful thinking by IT managers and consultants.
"Continue to provide people with what they need to succeed."
I suspect this management principle would also prove more effective at growing an organization than the Peter Principle.
Perhaps those old Pauls knew something that's still worth studying today.
"All new members in a hierarchical organization climb the hierarchy until they reach their level of maximum incompetence."
http://www.technologyreview.com/blog/arxiv/23800/
The Peter effect arises from the practice of promoting the best performer at Level N to a position at level N+1.
The Italians also simulated two other promotion policies:
1. Alternately promote first the most competent and then the least competent individuals.
2. Promote individuals at random.
According to their simulations, each of these methods improves the efficiency of an organization over the Peter method.
My thought: They could try promoting on the basis of who is most suited for the next level job. Duh!
Or maybe they could try not mixing the concept of "promotion" with that of "reward."
Or maybe even getting rid of the hierarchical notion altogether.
The Paul Principle
As a relevant post-script for my audience, they might want to look into the "Paul Principle," proposed by Paul Armer, who, like me, started out in computing as a desk calculator operator (or "computer" as we were known back then)."People become progressively less competent for jobs they once were well equipped to handle."
Paul proposed his law in 1970, the year after Peters proposed his. Paul claimed his principle was more relevant in high-tech fields, when the complexity of jobs grows faster than the people doing them. The Paul Principle has been virtually forgotten, but I think it is still worth some careful thinking by IT managers and consultants.
The Other Paul Principle
It seems there's another "Paul Principle," after St. Paul's treatise in Corinthians:"Continue to provide people with what they need to succeed."
I suspect this management principle would also prove more effective at growing an organization than the Peter Principle.
Perhaps those old Pauls knew something that's still worth studying today.
Friday, July 03, 2009
Why Choose One Conference Over Another?
In these days when money is short, lots of people cannot afford to participate all the conferences they might have attended last year. Money may be the first criterion for choosing conferences, but it's not the only one. I'm going to three conferences this year, each chosen by different characteristics. For me, money doesn't enter into it, so my reasons might be helpful for those who can afford to be in at least one conference, but are trying to choose. my first step in choosing a conference is to eliminate the majority of conferences by applying the following guides.
Few conferences meet all my criteria, but I look for those conferences that do, like the three (below) that I am attending this year. I have long-ago reached a stage in my life where I cannot tolerate several days sitting in an uncomfortable chair listening to someone read bullet points from PowerPoint slides.
CAST
I participated in the Conference of the Association for Software Testing (CAST) last year, and I'm returning this year because the subject of the conference is precisely focused on my current interest: promoting and improving the practice of software testing.
BizConf
BizConf is a new conference this year, and I'm participating primarily because of the other participants, who, like me, are small entrepreneurs running technology businesses. It's a small conference, limited to 75 participants, and scheduled once again to encourage spontaneous hall and meal sessions. All of the presenters I know are of the highest quality.
AYE (Amplifying Your Effectiveness)
You might say I participate in AYE every year because I'm a host--one of the people who created the conference. But I wouldn't have been a host in the first place if I had been satisfied with most of the conferences available. When we designed the conference, we had several issues in mind in addition to the ones listed above. We wanted the conference to be reasonably priced, easy to reach, and easy to learn more about than could be found on a simple website. We created a wiki that registrants could write on and anybody could read. We retained a small staff of intelligent, personable people (Lois and Suzy) to give information and solve problems over the phone. I hope we've made it easy to get to AYE, and I hope to see you there or at one of the other two conferences I'll be attending.
What Makes Conferences Less Attractive
I generally eliminate conferences that
- overschedule event with no time or place for spontaneous meetings
- overcrowd, usually to maximize profits, with just too many people, which encourages people to hang out only with their old pals
- lack adaptability so opportunities pass by without notice or care
- offer too much lecturing, not enough interaction, and insufficient experiential work--or none at all
- invite presenters of widely varied and untested skill and preparation
- do not name their presenters in advance, or give biographical information
- provide insufficient time and space for socializing, meeting new people
- allow little or no interaction with the presenters (In some conferences, presenters eat in a special area, intentionally separated from the participants. In others, presenters speak and run.
- schedule sales pitches instead of teaching presentations
- schedule canned pitches instead of original material
- offer too many plenary sessions, when participants have no choice of what to attend
Few conferences meet all my criteria, but I look for those conferences that do, like the three (below) that I am attending this year. I have long-ago reached a stage in my life where I cannot tolerate several days sitting in an uncomfortable chair listening to someone read bullet points from PowerPoint slides.
CAST
[http://www.associationforsoftwaretesting.org/drupal/CAST2009]
I participated in the Conference of the Association for Software Testing (CAST) last year, and I'm returning this year because the subject of the conference is precisely focused on my current interest: promoting and improving the practice of software testing.The sessions I attended were all of high quality and interest to me. Also, it's a reasonably small conference with numerous opportunities to participate in spontaneous hall sessions.
BizConf
[http://bizconf.heroku.com/]
BizConf is a new conference this year, and I'm participating primarily because of the other participants, who, like me, are small entrepreneurs running technology businesses. It's a small conference, limited to 75 participants, and scheduled once again to encourage spontaneous hall and meal sessions. All of the presenters I know are of the highest quality.AYE (Amplifying Your Effectiveness)
[http://www.ayeconference.com]
You might say I participate in AYE every year because I'm a host--one of the people who created the conference. But I wouldn't have been a host in the first place if I had been satisfied with most of the conferences available. When we designed the conference, we had several issues in mind in addition to the ones listed above. We wanted the conference to be reasonably priced, easy to reach, and easy to learn more about than could be found on a simple website. We created a wiki that registrants could write on and anybody could read. We retained a small staff of intelligent, personable people (Lois and Suzy) to give information and solve problems over the phone. I hope we've made it easy to get to AYE, and I hope to see you there or at one of the other two conferences I'll be attending.
Sunday, June 14, 2009
Was There Process Before Agile?
Reader June Kim recently wrote:
On p.328 of Volume 4 of your Quality Software Management series, the first line goes:
"Here's another example, mixing incremental development and hacking:"
In this Chapter 18, the process models you mention and describe are Waterfall, Cascade, Iterative Enhancement, and Prototyping (with Hacking and Rapid Prototyping as its variants). There isn't incremental development.
"incremental development" as you wrote on that page, doesn't appear before that line in the chapter.
Is it one among the four process models that you mentioned earlier in the chapter, or Is it something different?
ANSWER: First, you have to remember that when QSM was written, there was no "Agile" development craze. We were doing various development process, some of which were given capital letters, some which were not, and some which were "owned" by certain advocates. I was trying to be descriptive then, not favor anybody's pet process.
There were some "owned" processes (using the hated word, "methodology," and charging tens of thousands of dollars for shelves full of notebooks which nobody read). I suppose some of them are still around, but most of the organizations I work with are now smarter than to fall for that fallacy. For the most part, every organization custom-tailored its own process (or, in most cases, processes, plural).
Of course, that's still true today. I don't find many organizations using some "pure" version of agile.
ANSWER: Historically, the people who first named "Agile" processes were borrowing the best of all these methods. You could also say that any agile process is a cascade (or iterative enhancement if you're actually putting each iteration's output into use). Agile is much more than these processes, making explicit many team practices to support the iterative nature of the development.
ANSWER: That's always the case, no matter what the method, if the customer is reluctant to participate. (Until the end, when they whine, "But that's not what we wanted.")
QUESTION: What could you do in this case? Drop Agile process?
ANSWER: It's not an Agile process if the customer (or surrogate) isn't participating. In fact, I would drop any customer who doesn't participate. That's the rule I use in all my consulting, too. I don't believe you can help people who aren't willing to help themselves.
QUESTION: Or, make the customer be the part of the process? Then how? This is still a hard question to me, even with 10 years of experience in agile.
ANSWER: It's definitely one of the hardest questions with Agile or any method. Hard for most technical leaders because they lack the training and skills to work with reluctant customers.
So, I train them in these skills (a major goal of the AYE Conference and the PSL workshop), but primarily everything starts by simply pausing the work unless and until the customer has been identified and persuaded to participate.
On p.328 of Volume 4 of your Quality Software Management series, the first line goes:
"Here's another example, mixing incremental development and hacking:"
In this Chapter 18, the process models you mention and describe are Waterfall, Cascade, Iterative Enhancement, and Prototyping (with Hacking and Rapid Prototyping as its variants). There isn't incremental development.
"incremental development" as you wrote on that page, doesn't appear before that line in the chapter.
Is Agile An Example of Incremental Development?
QUESTION: Which process model are you referring to when you wrote "incremental development"?Is it one among the four process models that you mentioned earlier in the chapter, or Is it something different?
ANSWER: First, you have to remember that when QSM was written, there was no "Agile" development craze. We were doing various development process, some of which were given capital letters, some which were not, and some which were "owned" by certain advocates. I was trying to be descriptive then, not favor anybody's pet process.
There were some "owned" processes (using the hated word, "methodology," and charging tens of thousands of dollars for shelves full of notebooks which nobody read). I suppose some of them are still around, but most of the organizations I work with are now smarter than to fall for that fallacy. For the most part, every organization custom-tailored its own process (or, in most cases, processes, plural).
Of course, that's still true today. I don't find many organizations using some "pure" version of agile.
Where Do You Place Agile?
QUESTION: I am interested where you would put Agile process. I think Agile(XP and Scrum, for example) is closest to Rapid Prototyping as you described.ANSWER: Historically, the people who first named "Agile" processes were borrowing the best of all these methods. You could also say that any agile process is a cascade (or iterative enhancement if you're actually putting each iteration's output into use). Agile is much more than these processes, making explicit many team practices to support the iterative nature of the development.
What Happens When the Customer Won't Participate?
QUESTION: If so, it has the same danger when the customer isn't willing to be the integral part of the process.ANSWER: That's always the case, no matter what the method, if the customer is reluctant to participate. (Until the end, when they whine, "But that's not what we wanted.")
QUESTION: What could you do in this case? Drop Agile process?
ANSWER: It's not an Agile process if the customer (or surrogate) isn't participating. In fact, I would drop any customer who doesn't participate. That's the rule I use in all my consulting, too. I don't believe you can help people who aren't willing to help themselves.
QUESTION: Or, make the customer be the part of the process? Then how? This is still a hard question to me, even with 10 years of experience in agile.
ANSWER: It's definitely one of the hardest questions with Agile or any method. Hard for most technical leaders because they lack the training and skills to work with reluctant customers.
So, I train them in these skills (a major goal of the AYE Conference and the PSL workshop), but primarily everything starts by simply pausing the work unless and until the customer has been identified and persuaded to participate.
Friday, June 05, 2009
Session Based Test Management Advice
One of the challenges of a blog on consulting is the difficulty of showing what actual consulting looks like. From time to time, though, I receive a consulting request by email which can be answered with a brief email. Brent Plumley recently sent me a request and had given me permission to blog it and my reply, so others may share, and comment.
- I have several areas that I was hoping you would be able to provide some insight
(1) Testing Session Debrief Trade off
- When a test session is reviewed we consider the following and attempt to achieve a 10/10 score on each.
(a) Quality of testing performed
- Did all the risk areas get covered
- Were the correct test techniques implemented
- Are there anything missing(different combinations, order of events etc...)
(b) Quality of Testing Notes
- Do the notes clearly outline the testing
- Is the testing reproducible using the notes
- Does the test properly convey the thoughts and observations
- etc...
- As a result, the time required to perform the debriefs is quite long.
QUESTION: When performing a session debriefs should we consider a trade off between 'time', 'Quality of Testing Performed' and 'Quality of Testing Notes'. (ie. spent less time, ensure 10/10 on 'quality of testing performed' and 6/10 on 'quality of notes') or, do you think that both (a) and (b) above are important?
(2) Debrief Process
- Even if we can reduce the time spent on session reviews, as the team continues to grow, we will once again be faced with the problem of scalability (more testers = more test sessions = more time needed to review)
- We are currently considering several ways to address the scalability issue
(1) Test Sessions are reviewed by other team members (peer-review process). This will help relieve the strain on the Test Manager of having to review all test session. Also, this will provide opportunity to all test team members to learn from each others unique styles.
ANSWER: This is what I usually suggest to clients in similar situations. The number one job of any manager is to develop the people who work under their mentorship. This is an excellent way to do so.
You can introduce this leadership training opportunity incrementally, with the early leads being supervised by the manager until she finds each person prepared to lead on their own.
(2) Test Manager continues to review the test sessions but depending on the experience level of the tester, only a certain percentage of test sessions will be reviewed (ex: 100% of sessions are reviewed for junior/New testers, and 15% of test sessions for more experienced testers)
ANSWER: Absolutely not. This simply encourages people to game the system by not having their test sessions reviewed. DO NOT DO THIS.
- I have identified several problems with both of the above options. For both options, the test manager may feel out of the loop as much of the testing is not reviewed by them. Ultimately it is the test manager that is responsible for signing off on the testing that is performed. If the session notes are reviewed by peers, or not reviewed at all (only 15% of experienced testers sessions are reviewed), the test manager personally cannot guarantee perfect coverage of testing.
ANSWER: Managers must know how to delegate. If you cannot delegate, you should not be a manager.
Option 2, however, is not a matter of delegating, but of abdicating.
Again, DO NOT DO THIS.
- We are currently leading towards the Peer-Review process. This will ensure that all test sessions are reviewed by someone, which will help guarantee proper test coverage.
ANSWER: I totally agree.
QUESTION: Do you know of any other debrief processes which have been successful for other companies?
ANSWER: Not successful ones.
QUESTION: What method would you suggest for keeping the test manager informed and in the loop? We have thought of bi-weekly team meetings where key points and high level outline of testing performed is presented. Of another option, the test manager will debrief a small sample of the test sessions to get a measure of the quality of testing and debriefing performed by their team.
ANSWER: Forget the sampling. Each person who leads a review should fill out a simple report that everyone, including the test manager, can see. (For more on such reports, see my book, The Handbook of Walkthroughs, Inspections, and Technical Reviews.)
I hope this helps, Brent. It represents half a century of observations on my clients, what works and what doesn't work.
My blog readers, of course, know that no advice works in every situation, though certain mistakes seem to repeat themselves with every new generation. Brent responded by telling me the advice was helpful to him, but let me know how this advice works for you. And ask for answers and clarification.
Brent's Situation
I am currently researching methods to improve our (Session Based Test Management) SBTM debrief/review process in an attempt to make the process more efficient and scalable. Our current process has the Test Manager reviewing 100% of the testing sessions. As the testing team grows, the test manager no longer has the capacity to review all the sessions and as a result a significant backlog session list has accumulated.- I have several areas that I was hoping you would be able to provide some insight
(1) Testing Session Debrief Trade off
- When a test session is reviewed we consider the following and attempt to achieve a 10/10 score on each.
(a) Quality of testing performed
- Did all the risk areas get covered
- Were the correct test techniques implemented
- Are there anything missing(different combinations, order of events etc...)
(b) Quality of Testing Notes
- Do the notes clearly outline the testing
- Is the testing reproducible using the notes
- Does the test properly convey the thoughts and observations
- etc...
- As a result, the time required to perform the debriefs is quite long.
QUESTION: When performing a session debriefs should we consider a trade off between 'time', 'Quality of Testing Performed' and 'Quality of Testing Notes'. (ie. spent less time, ensure 10/10 on 'quality of testing performed' and 6/10 on 'quality of notes') or, do you think that both (a) and (b) above are important?
(2) Debrief Process
- Even if we can reduce the time spent on session reviews, as the team continues to grow, we will once again be faced with the problem of scalability (more testers = more test sessions = more time needed to review)
- We are currently considering several ways to address the scalability issue
(1) Test Sessions are reviewed by other team members (peer-review process). This will help relieve the strain on the Test Manager of having to review all test session. Also, this will provide opportunity to all test team members to learn from each others unique styles.
ANSWER: This is what I usually suggest to clients in similar situations. The number one job of any manager is to develop the people who work under their mentorship. This is an excellent way to do so.
You can introduce this leadership training opportunity incrementally, with the early leads being supervised by the manager until she finds each person prepared to lead on their own.
(2) Test Manager continues to review the test sessions but depending on the experience level of the tester, only a certain percentage of test sessions will be reviewed (ex: 100% of sessions are reviewed for junior/New testers, and 15% of test sessions for more experienced testers)
ANSWER: Absolutely not. This simply encourages people to game the system by not having their test sessions reviewed. DO NOT DO THIS.
- I have identified several problems with both of the above options. For both options, the test manager may feel out of the loop as much of the testing is not reviewed by them. Ultimately it is the test manager that is responsible for signing off on the testing that is performed. If the session notes are reviewed by peers, or not reviewed at all (only 15% of experienced testers sessions are reviewed), the test manager personally cannot guarantee perfect coverage of testing.
ANSWER: Managers must know how to delegate. If you cannot delegate, you should not be a manager.
Option 2, however, is not a matter of delegating, but of abdicating.
Again, DO NOT DO THIS.
- We are currently leading towards the Peer-Review process. This will ensure that all test sessions are reviewed by someone, which will help guarantee proper test coverage.
ANSWER: I totally agree.
QUESTION: Do you know of any other debrief processes which have been successful for other companies?
ANSWER: Not successful ones.
QUESTION: What method would you suggest for keeping the test manager informed and in the loop? We have thought of bi-weekly team meetings where key points and high level outline of testing performed is presented. Of another option, the test manager will debrief a small sample of the test sessions to get a measure of the quality of testing and debriefing performed by their team.
ANSWER: Forget the sampling. Each person who leads a review should fill out a simple report that everyone, including the test manager, can see. (For more on such reports, see my book, The Handbook of Walkthroughs, Inspections, and Technical Reviews.)
I hope this helps, Brent. It represents half a century of observations on my clients, what works and what doesn't work.
My blog readers, of course, know that no advice works in every situation, though certain mistakes seem to repeat themselves with every new generation. Brent responded by telling me the advice was helpful to him, but let me know how this advice works for you. And ask for answers and clarification.
Sunday, May 17, 2009
Why We Love and Hate Meetings
Did you ever notice how many consultants have back problems? I do, from too much time in miserable seats on airplane, working on my computers at home, or sitting in boring meetings at clients' offices.
Because of my back problems, I can't bend over easily, which means I can't do an effective job of cutting my toenails. So, when I need to trim my toenails, I visit a salon for a pedicure. While having my toes clipped, I read the available magazines, such as Brides, Modern Bride, and Elegant Bride.
Bridal magazines are incredibly popular, with 135 different ones in print last time I checked. Topics include Beach Weddings, Bridal Parties, Destination Weddings, Accessories, Cakes, Ceremonies, Decorations, Dresses, Etiquette, Favors, Flowers, Gifts, Invitations, Planners, Receptions, Traditions, Trends, and many others.
Looking at all these magazines, I asked myself, "Who reads this stuff?" Well, obviously, guys don't usually read it, because nothing in the magazines evokes any emotional response—in guys. For many women, it's a different story.
From writing fiction, I've learned that emotion sells writing. My Plotbusters critique group constantly tells me that my stories don't sufficiently describe or evoke strong emotions. I didn't understand their comments, because the rest of my reader network didn't agree with them. What was the difference?
My Plotbusters colleagues are all multi-published writers, but generally not techies like the rest of my reader network. Evidently, non-techies don't fully appreciate my fiction. They say, "It's not emotional enough." Why? Because mostly the stories do not involve conflict, random violence, death, bad sex, unrequited love, and so forth.
What they do involve is smart people trying to solve problems, step by step. To me, that's not boring at all, but, indeed, extremely emotional. Tremendously exciting—which perhaps defines me as a nerd.
Which brings me back to meetings—why they bore me and cause back problems.
When I walk a client's corridors, I frequently meet people on their way to meetings. Although these same people have told me that meetings are boring, they often seem excited when they're on their way to one. Why?
Over the years, here's what I figured out. Most of my clients' people are techies—nerds like me. They find the meetings boring when they don't seem to be trying to solve problems, step by step.
At the same time, what these meetings are doing is playing out an emotional drama—conflict, blaming, flirting, one-upsmanship, random outbursts, anger, and so forth. For these happy people heading for meetings, it's those the soap-opera aspects of meetings are the most exciting parts of their jobs.
To the techies, the interest in these soap-operas is a distant second to the interest in a well-conducted problem-solving session. On the other hand, all this drama—the stuff we contemptuously call "politics"—seems to be the bread and butter of the non-techies. Indeed, these people are often upset if I show them how to conduct well-run meetings, because I've taken all the joy out of their lives.
Maybe I should bring them Brides Magazine to read in the time they save. Or perhaps Hot Rod Magazine for the guys.
Oh, and BTW, if you like to read stories of smart people trying to solve problems and be happy, take a look at my eStore.
Saturday, April 11, 2009
Reader Feedback: Sample Pages
If you want to have a successful business, you have to pay attention to customer feedback. In the case of eMarketing, that feedback can be fast and focused. Using Google Analytics, I discovered my eStore was attracting 50 or more readers a day, but most these potential customers were not buying books.
Following some astute reader feedback, I've now made sample pages available for each of the novels in my eStore:
Mistress of Molecules
First Stringers: or eyes that do not see
The Freshman Murders
The Hands of God
This way, my readers will not be buying a pig in a poke. We'll see if it works.
Following some astute reader feedback, I've now made sample pages available for each of the novels in my eStore:
Mistress of Molecules
First Stringers: or eyes that do not see
The Freshman Murders
The Hands of God
This way, my readers will not be buying a pig in a poke. We'll see if it works.
Thursday, April 02, 2009
eNovel Store: First Month Report and Lesson
It's been about a month since I opened my eNovel store. After a snappy first two weeks, activity slowed to zero, and I thought nobody would ever buy another novel of mine, let alone put me over the tipping point where the novels would take off like Harry Potter.
I learned that worrying over day-to-day or week-to-week sales is a futile wasted of time. After one dry week, sales picked up again. If they stay at this level, I'll earn a small but welcome addition for my charities each year.
Will I be satisfied? I don't know, but at least I won't worry day-to-day. I've learned my lesson.
It's a worthwhile lesson for consultants in all phases of their business. By its nature, independent consulting is a highly variable business. As I wrote in The Secrets of Consulting, there are, theoretically, three states a consultant's business can be in: A: too much business; B: not enough business; and C:just the right amount of business. But no individual is ever in state C.
So stop worrying and do something about it. Me, I'm asking everyone I know to take a look at my book store.
I learned that worrying over day-to-day or week-to-week sales is a futile wasted of time. After one dry week, sales picked up again. If they stay at this level, I'll earn a small but welcome addition for my charities each year.
Will I be satisfied? I don't know, but at least I won't worry day-to-day. I've learned my lesson.
It's a worthwhile lesson for consultants in all phases of their business. By its nature, independent consulting is a highly variable business. As I wrote in The Secrets of Consulting, there are, theoretically, three states a consultant's business can be in: A: too much business; B: not enough business; and C:just the right amount of business. But no individual is ever in state C.
So stop worrying and do something about it. Me, I'm asking everyone I know to take a look at my book store.
Wednesday, March 11, 2009
eNovels for Nerds
I've decided it's past time for me to join the electronic age.
Five years ago, I gave myself the assignment of spending five years learning to write the kind of fiction that would further my life's goal:
helping smart people be happy
I have achieved my goal, and have now completed ten novels, one of which has been published as The Aremac Project. All of the novels are written about smart people and their struggles to be happy. They're intended to make the learnings from my other books and my workshops come to life in a new medium. So far, they seem to be working that way.
I've achieved something else besides my original goal: I have learned more than I wanted to know about the fiction publishing business.
Most of all, I've learned how hard and slow it the business is, and how difficult it is to break into. For example, one of my fellow writers just sent me statistics on his responses to queries about his crime thriller: 93% of the editors simply did not bother to reply, even with a short email. My own success rate (at just receiving a reply, even in an enclosed, stamped, self-addressed envelope) has only slightly better.
From the editors' point of view, there are good reasons for this rude-looking behavior: they are simply swamped with manuscripts and queries from would-be authors. And, even when they do reply, and even when they do accept the manuscript for publication (a much lower percentage still), they usually require years to get the novel in print. And, once it's in print, chances are it won't stay in print for more than a year or so.
I've decided I'm too old to put all my chips in that game. I'll still seek print publication by some publishers, while at the same time, I'm publishing some of my novels in eBook format (pdf) sold through my website for the value price of $4.99 apiece. I invite you to visit the site and try one of them. (I always give a money-back guarantee on all my work.)
I would love to have your feedback on the store itself, as well as the form and content of the novels. If this approach is well-received, I'll put some more of my novels up there.
So, visit http://www.geraldmweinberg.com/Site/eBOOKS.html and take a look. As time goes by, I'll report here on the outcome of this experiment.
Five years ago, I gave myself the assignment of spending five years learning to write the kind of fiction that would further my life's goal:
helping smart people be happy
I have achieved my goal, and have now completed ten novels, one of which has been published as The Aremac Project. All of the novels are written about smart people and their struggles to be happy. They're intended to make the learnings from my other books and my workshops come to life in a new medium. So far, they seem to be working that way.
I've achieved something else besides my original goal: I have learned more than I wanted to know about the fiction publishing business.
Most of all, I've learned how hard and slow it the business is, and how difficult it is to break into. For example, one of my fellow writers just sent me statistics on his responses to queries about his crime thriller: 93% of the editors simply did not bother to reply, even with a short email. My own success rate (at just receiving a reply, even in an enclosed, stamped, self-addressed envelope) has only slightly better.
From the editors' point of view, there are good reasons for this rude-looking behavior: they are simply swamped with manuscripts and queries from would-be authors. And, even when they do reply, and even when they do accept the manuscript for publication (a much lower percentage still), they usually require years to get the novel in print. And, once it's in print, chances are it won't stay in print for more than a year or so.
I've decided I'm too old to put all my chips in that game. I'll still seek print publication by some publishers, while at the same time, I'm publishing some of my novels in eBook format (pdf) sold through my website for the value price of $4.99 apiece. I invite you to visit the site and try one of them. (I always give a money-back guarantee on all my work.)
I would love to have your feedback on the store itself, as well as the form and content of the novels. If this approach is well-received, I'll put some more of my novels up there.
So, visit http://www.geraldmweinberg.com/Site/eBOOKS.html and take a look. As time goes by, I'll report here on the outcome of this experiment.
Friday, February 27, 2009
My Favorite Bug
Michael Hunter asked me to write something for his blog about "my favorite bug." At first I thought it was a silly assignment, but, as usual, once I sat down to write, I learned a few things. I don't know when (or if) it will appear on his blog, so I'm putting it here because I think it's too good not to share. (If you think I lack humility, consider the subject: how stupid I can be.)
It was the first commercial program I had ever written--a pipeline network analyzer for municipal water systems. (note 1) Essentially, it solved systems of non-linear algebraic equations, by iteration.
It passed all my prepared tests (test first, back in 1956), so we brought in civil engineers with data from a city near San Francisco (not San Jose, which barely existed back then). We set up and started to run the first set of data. Then we waited while the IBM 650 ground away. Thirty minutes--surpassing my largest test case. One hour--surpassing my wildest imagination. Two hours--making me suspect the program was in a closed loop.
I got up from the table where we'd been checking the input data. I was about to stop the machine and find out where the program had gone wrong, but before I reached the machine, it began to punch cards. (We had no on-line printing in those days.) The cards contained the solution, exactly according to spec.
I fixed the bug by having the program punch out a "progress card" after every complete iteration. The card contained a set of numbers that characterized the convergence so far. From the sequence of cards, we could observe the rate of convergence and decide if we wanted the program to continue. If we wanted to stop, we could set a switch on the console and force the program to punch out the full set of values so far--in the input format, so we could resume later, if we wished. (Having no breakpoint in a program that could run twenty hours was another design bug.)
a. the relationship between a program and its user
b. the potential performance characteristics of the program
c. the possibility that an error might occur (hardware or software or human) that could cost us thousands of dollars of wasted computer and human time.
d. that I ought to be more humble about my programming skills. Before that, I had never made an error in one of perhaps ten small programs I had written, and I thought I was pretty terrific. This humiliating experience made me appreciative of a good testing process (We didn't have "testers" in those days. We didn't even have "programmers." My title was "Applied Science Representative."
This was the beginning of a long career of learning about programmers, programming, and testing. Many of those learnings are captured in my book, Perfect Software and Other Illusions About Testing (note 2)
Well, it wasn't my last bug, but it was first, and my best.
(note 2) Perfect Software (And Other Illusions About Testing)
http://www.dorsethouse.com/savings.html#perf
My First Commercial Program
My favorite bug is also my first bug. It's my favorite because it started me on the road to learning the most important things about programming and testing.It was the first commercial program I had ever written--a pipeline network analyzer for municipal water systems. (note 1) Essentially, it solved systems of non-linear algebraic equations, by iteration.
It passed all my prepared tests (test first, back in 1956), so we brought in civil engineers with data from a city near San Francisco (not San Jose, which barely existed back then). We set up and started to run the first set of data. Then we waited while the IBM 650 ground away. Thirty minutes--surpassing my largest test case. One hour--surpassing my wildest imagination. Two hours--making me suspect the program was in a closed loop.
I got up from the table where we'd been checking the input data. I was about to stop the machine and find out where the program had gone wrong, but before I reached the machine, it began to punch cards. (We had no on-line printing in those days.) The cards contained the solution, exactly according to spec.
So, what was the bug?
There was nothing wrong with the computation, but the design of the human interface was bad, bad, bad. I had not estimated the performance characteristics as the number of equations grew. I had not accounted for human patience (or impatience) and tolerance (or intolerance) for uncertainty. The case that "looped" was the smallest of our cases. The largest would have run something like twenty hours.I fixed the bug by having the program punch out a "progress card" after every complete iteration. The card contained a set of numbers that characterized the convergence so far. From the sequence of cards, we could observe the rate of convergence and decide if we wanted the program to continue. If we wanted to stop, we could set a switch on the console and force the program to punch out the full set of values so far--in the input format, so we could resume later, if we wished. (Having no breakpoint in a program that could run twenty hours was another design bug.)
Learnings
After than experience, I've never failed to considera. the relationship between a program and its user
b. the potential performance characteristics of the program
c. the possibility that an error might occur (hardware or software or human) that could cost us thousands of dollars of wasted computer and human time.
d. that I ought to be more humble about my programming skills. Before that, I had never made an error in one of perhaps ten small programs I had written, and I thought I was pretty terrific. This humiliating experience made me appreciative of a good testing process (We didn't have "testers" in those days. We didn't even have "programmers." My title was "Applied Science Representative."
This was the beginning of a long career of learning about programmers, programming, and testing. Many of those learnings are captured in my book, Perfect Software and Other Illusions About Testing (note 2)
Well, it wasn't my last bug, but it was first, and my best.
Notes
(note 1) Weinberg, Gerald M., and Lyle N. Hoag. "Pipeline Network Analysis by Electronic Digital Computer." Journal of the American Water Works Association 49, no. 5 (1957).(note 2) Perfect Software (And Other Illusions About Testing)
http://www.dorsethouse.com/savings.html#perf
Monday, February 23, 2009
Three Lessons from a Thirty-Year Bug
Reader Michael Bolton writes:
I'm reading General Principles of Systems Design, and enjoying it. I'm confused by something, and I think it's because of an error in the text.On page 106, there's a matrix that is intended to describe the bathtubs illustrated in Figure 5.1 and diagrammed in figure 5.2. My interpretation is that the last row of the matrix should read
0 0 1 1 0
The text suggests
0 1 1 1 0
I interpret this as meaning that bathtub 1 could supply water directly to K, the sink; but neither Figure 5.1 nor 5.2 suggest that. Am I misunderstanding, or is there an error?
My response
It's an error in the text, previously unreported.Seems as though it's been sitting there for 30 years and tens of thousands of readers.
Moral Number One:
Several morals, but to me, the most important one is about testability. Figures 5.1 and 5.2 are in the previous chapter, and difficult to look at while you're looking at the matrix. This makes testing quite difficult. I should have repeated one of the figures so it appeared on the same page as the matrix.Moral Number Two:
In writing, as in software development, there's no such thing as perfection. (For more on this subject, see my book, Perfect Software, and other illusions about testing.) Just because nobody's found a bug for 29 years doesn't mean one won't turn up in year 30. If you start believing in perfection, you may be in for a nasty shock.Moral Number Three
: It would have been easy to blame my readers for being careless, inattentive, or just plain dumb not to have detected and reported this bug (which actually appears twice). If I did that, however, I would have shielded myself from learning the first moral, which puts the responsibility squarely on me. If you don't take responsibility for your mistakes, learning doesn't happen.Slow Learning is Better than No Learning
So, that's three lessons in thirty years. I'm a pretty slow learner, but at least three is better than zero. So, I'm going to be proud of myself for learning at all.Thursday, February 19, 2009
A Life-Changing Experience for You
In one month, Esther, Johanna, and I will be starting our next Problem Solving Leadership (PSL) workshop here in Albuquerque. Thus, the timing was just right for several new blog posts about people's experience in PSL.
First to appear was David Barnholdt's blog entry about what happened for him at the most recent PSL, in Sweden. (Read David's post.)
Then, today, Selena Delesie's post about her reactions two years after her PSL experience here in New Mexico. (Read Selena's post.)
And then David followed up with a post about an extremely successful exercise he designed for his own courses, based on his learnings in PSL. (Read about David's exercise.)
So, why am I suggesting you visit these blogs? Nothing to hide: I'm trying to convince you to come participate in PSL in March. I know times are hard for many of us, but hard times are just the times we most need life-changing experiences like Selena and David and many others have had.
You can find out more about PSL at Esther's website, and contact Esther Derby or me or Johanna Rothman
Also, if you know of any other blogs about PSL experiences, let me know, and I'll add them to this post. I'll do almost anything to encourage people to join us in the March PSL.
Added on 27 February 2009
Henrik Kniberg was in the Sweden PSL class and wrote another blog entry building on David's experiential exercise.
First to appear was David Barnholdt's blog entry about what happened for him at the most recent PSL, in Sweden. (Read David's post.)
Then, today, Selena Delesie's post about her reactions two years after her PSL experience here in New Mexico. (Read Selena's post.)
And then David followed up with a post about an extremely successful exercise he designed for his own courses, based on his learnings in PSL. (Read about David's exercise.)
So, why am I suggesting you visit these blogs? Nothing to hide: I'm trying to convince you to come participate in PSL in March. I know times are hard for many of us, but hard times are just the times we most need life-changing experiences like Selena and David and many others have had.
You can find out more about PSL at Esther's website, and contact Esther Derby or me or Johanna Rothman
Also, if you know of any other blogs about PSL experiences, let me know, and I'll add them to this post. I'll do almost anything to encourage people to join us in the March PSL.
Added on 27 February 2009
Henrik Kniberg was in the Sweden PSL class and wrote another blog entry building on David's experiential exercise.
Monday, February 09, 2009
Problem Solving Leadership Workshop
We're giving the next offering of the famous Problem Solving Leadership (PSL) on March 22-27, 2009 Albuquerque, NM led by me, Esther Derby, and Johanna Rothman
The workshop's purpose is to learn and practice a consultant's most valuable asset: the ability to think and act creatively. We have designed this workshop to be practical and applicable to the modern workplace. Your problems and concerns provide a frame of reference for all
the workshop activities.
What you will learn
. to be a leader while being a member of a team
. to focus your thinking while in chaos
. to make change a productive, creative event
. to build truly effective teams
. to design projects people really want to work on
. to observe exactly what is happening
. to use tools of effective communication
. to handle conflict in problem solving groups
The workshop provides five and a half days of intensive focus on developing your unique consulting style and abilities.
If you would like more information about in this unique workshop experience, send email to
jr@jrothman.com and/or take a look at http://www.estherderby.com/workshops/ProblemSolvingLeadership.htm
The workshop's purpose is to learn and practice a consultant's most valuable asset: the ability to think and act creatively. We have designed this workshop to be practical and applicable to the modern workplace. Your problems and concerns provide a frame of reference for all
the workshop activities.
What you will learn
. to be a leader while being a member of a team
. to focus your thinking while in chaos
. to make change a productive, creative event
. to build truly effective teams
. to design projects people really want to work on
. to observe exactly what is happening
. to use tools of effective communication
. to handle conflict in problem solving groups
The workshop provides five and a half days of intensive focus on developing your unique consulting style and abilities.
If you would like more information about in this unique workshop experience, send email to
jr@jrothman.com and/or take a look at http://www.estherderby.com/workshops/ProblemSolvingLeadership.htm
Saturday, February 07, 2009
Estimating (continued)
Back on April 17, 2006, I posted an entry called "Estimating Projects: A History of Failure." I try to make entries that are not ephemeral, so I'm happy to see that people are still commenting on this one almost three years later. I'm suggesting here that you go back on the blog and read the latest comment, from "Will."
Will has many wise things to say about how to do, and not do, estimating. Will, if you're writing a book on project management, give us the title, publisher, and estimated date we can expect to see it.
Take a look, and if you have comments, add them here.
Will has many wise things to say about how to do, and not do, estimating. Will, if you're writing a book on project management, give us the title, publisher, and estimated date we can expect to see it.
Take a look, and if you have comments, add them here.
Monday, January 05, 2009
Testability and Reproducibility
Doug Szabo has asked me two interesting questions, and we thought we would share my answers.
Doug: Can you point me to some guiding literature that explains how to make code testable?
Explanation: I noticed at least one mention in Perfect Software: and Other Illusions about Testing of making code testable. I have seen that language in a couple of other software testing books, including Software Testing Techniques. I have even been told by more than one developer that they needed to "make the code more testable" before they wanted me to start testing. Since the developers who told me this didn't have the faintest clue what testing was about, I really don't know what they intended to do with their code. Apparently, neither did they - I eventually asked, "what do you need to do to make it more testable?", to which they replied that they didn't know. I was and still am confounded by the lack of explanation for what it is that makes code testable. I understand when code is very difficult to test, like when you have a "horrible loop" (Software Testing Techniques), or when multithreaded code is not first tested in single thread contexts, but I don't understand what needs to be done to code to make it testable. Are we talking about instrumenting the code with Debug statements, assert statements, or some symbol that a test tool can detect?
Jerry: I share your dismay at the lack of publications about how to make code testable. There are many, many little techniques (like initialization on entry, not exit; eliminating as many special cases as possible; and general simplification for a reader), but they're not as important as three things:
1. All code should be open code that anyone in the project can read and critique.
2. All code must be reviewed in a technical review (see my The Handbook of Walthroughs, Inspections, and Technical Reviews)--in which at one professional tester is present and fully participating.
3. Same as 2, except for design reviews and requirements reviews).
If you do these thing, the organization will quickly learn how to make code testable.
But, yes, someone should write the book and start with these three things.
Q2: Do you have some strategies for triaging bugs that do not reproduce consistently?
Explanation: I was a developer before my current role of tester. Hey. Don't roll your eyes at me. I was an engineer (a real one, with a professional license and all that hoopla) before I became a developer. So I already knew the value of testing, even if I didn't know what software testing really was. Well, as an engineer it was crucial that tests be designed such that the results would be reproducible, and measurable against a control. When I got into software development, I tried to stick to the engineering principle with respect to testing. Unfortunately, as I worked on larger software projects, particularly those where my code talked to other processes, and also where I had multiple threads running, I found that bugs were starting to occur where steps to reproduce did not consistently reproduce the bug. Oh oh. I knew that meant there was something wrong, I just didn't have something that I could run under a debugger and know where to set a breakpoint. As a developer, it seemed like there were always enough reproducible bugs that I had lots of excuses to avoid trying to solve those that might not reproduce. Now, as a tester, I am empathetic to developers and have a self-imposed guideline to make an entry of any non-reproducible issue, but at the same time I don't assign it to the pool of issues to fix until a set of steps to consistently reproduce is found. What I got out of Perfect Software is that perhaps I should be passing the tough to reproduce issues over to the pool to fix, but then what would you recommend for a triage approach, to convince stakeholders to take those issues as seriously as the ones that do consistently reproduce?
Jerry: Great question, Doug.
First answer. Try changing "not reproducible" to "I am unable to reproduce."
Then parse the second category into sub-categories like:
a. I saw this one, but was never able to make it happen again.
b. I see this when I run with this setup, but sometimes I don't.
c. This is the same anomaly I see under several setups, but not all the time.
d. Under X setup, this happens sometimes and not other times. There may be something different from one X to the other, but I' not seeing it.
In each case, you are unable to pinpoint the fault underlying the failure. There may be several faults producing the same failure, or different failures appearing from the same underlying fault. Since you don't really know the frequency of this failure, the way I use to triage the failure is to apply the question: "What would it cost us every time this failure appears in the field?"
If that number is high, then get a team working on the bug. If it's low, let the bug rest while (perhaps) new data accumulates or the bug disappears mysteriously as other changes are made to the code.
Doug: Can you point me to some guiding literature that explains how to make code testable?
Explanation: I noticed at least one mention in Perfect Software: and Other Illusions about Testing of making code testable. I have seen that language in a couple of other software testing books, including Software Testing Techniques. I have even been told by more than one developer that they needed to "make the code more testable" before they wanted me to start testing. Since the developers who told me this didn't have the faintest clue what testing was about, I really don't know what they intended to do with their code. Apparently, neither did they - I eventually asked, "what do you need to do to make it more testable?", to which they replied that they didn't know. I was and still am confounded by the lack of explanation for what it is that makes code testable. I understand when code is very difficult to test, like when you have a "horrible loop" (Software Testing Techniques), or when multithreaded code is not first tested in single thread contexts, but I don't understand what needs to be done to code to make it testable. Are we talking about instrumenting the code with Debug statements, assert statements, or some symbol that a test tool can detect?
Jerry: I share your dismay at the lack of publications about how to make code testable. There are many, many little techniques (like initialization on entry, not exit; eliminating as many special cases as possible; and general simplification for a reader), but they're not as important as three things:
1. All code should be open code that anyone in the project can read and critique.
2. All code must be reviewed in a technical review (see my The Handbook of Walthroughs, Inspections, and Technical Reviews)--in which at one professional tester is present and fully participating.
3. Same as 2, except for design reviews and requirements reviews).
If you do these thing, the organization will quickly learn how to make code testable.
But, yes, someone should write the book and start with these three things.
Q2: Do you have some strategies for triaging bugs that do not reproduce consistently?
Explanation: I was a developer before my current role of tester. Hey. Don't roll your eyes at me. I was an engineer (a real one, with a professional license and all that hoopla) before I became a developer. So I already knew the value of testing, even if I didn't know what software testing really was. Well, as an engineer it was crucial that tests be designed such that the results would be reproducible, and measurable against a control. When I got into software development, I tried to stick to the engineering principle with respect to testing. Unfortunately, as I worked on larger software projects, particularly those where my code talked to other processes, and also where I had multiple threads running, I found that bugs were starting to occur where steps to reproduce did not consistently reproduce the bug. Oh oh. I knew that meant there was something wrong, I just didn't have something that I could run under a debugger and know where to set a breakpoint. As a developer, it seemed like there were always enough reproducible bugs that I had lots of excuses to avoid trying to solve those that might not reproduce. Now, as a tester, I am empathetic to developers and have a self-imposed guideline to make an entry of any non-reproducible issue, but at the same time I don't assign it to the pool of issues to fix until a set of steps to consistently reproduce is found. What I got out of Perfect Software is that perhaps I should be passing the tough to reproduce issues over to the pool to fix, but then what would you recommend for a triage approach, to convince stakeholders to take those issues as seriously as the ones that do consistently reproduce?
Jerry: Great question, Doug.
First answer. Try changing "not reproducible" to "I am unable to reproduce."
Then parse the second category into sub-categories like:
a. I saw this one, but was never able to make it happen again.
b. I see this when I run with this setup, but sometimes I don't.
c. This is the same anomaly I see under several setups, but not all the time.
d. Under X setup, this happens sometimes and not other times. There may be something different from one X to the other, but I' not seeing it.
In each case, you are unable to pinpoint the fault underlying the failure. There may be several faults producing the same failure, or different failures appearing from the same underlying fault. Since you don't really know the frequency of this failure, the way I use to triage the failure is to apply the question: "What would it cost us every time this failure appears in the field?"
If that number is high, then get a team working on the bug. If it's low, let the bug rest while (perhaps) new data accumulates or the bug disappears mysteriously as other changes are made to the code.
Subscribe to:
Posts (Atom)
