tag:blogger.com,1999:blog-25922407.post7704644917682987559..comments2024-03-25T22:12:49.064-07:00Comments on Gerald Weinberg's Secrets of Writing and Consulting: Testability and ReproducibilityGerald M. Weinberghttp://www.blogger.com/profile/05902673055244863609noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-25922407.post-38230976842368609252015-12-22T11:11:47.705-08:002015-12-22T11:11:47.705-08:00It occurred to me that I hadn't posted this bl...It occurred to me that I hadn't posted this blog post by James Bach, which might be helpful to answering the first of the original two questions.<br /><br />http://www.satisfice.com/blog/archives/34<br /><br />---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-33990486018638130962015-12-21T21:15:23.031-08:002015-12-21T21:15:23.031-08:00Late to the party, I know, but this may help someo...Late to the party, I know, but this may help someone one day.<br />Regarding the question of logging defects that you saw once and can not reproduce or are inconsistently replicable... Yes, log them.<br /><br />Part of the benefits of testing is reducing unknown risks for the stakeholder. The more defects they are aware of, the better they can assess their risks in accepting the software.<br /><br />In those instances where a defect is difficult to get to reproduce, additional commentary from the tester to explain the overall impact of the defect occurring should be added to the defect record, again, to allow the stakeholder to make a risk assessment and (hopefully) accept the defect into their UAT, pre-prod and/or production environment(s).DrummerDaveFhttps://www.blogger.com/profile/00911734126021147849noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-585310161080761262009-03-02T08:40:00.000-08:002009-03-02T08:40:00.000-08:00By the way - you can change how Blogger shows comm...By the way - you can change how Blogger shows comments to include the date: It's a settings in the "comments" section of your blog setting. Look for "Comments Timestamp Format"<BR/>There are about two dozen formats to choose from.Matisse Enzerhttps://www.blogger.com/profile/03736762585596345292noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-73600446253452590222009-03-02T08:35:00.000-08:002009-03-02T08:35:00.000-08:00> the lack of publications about how to make co...> the lack of publications about how to make code testable.<BR/><BR/>There are, I believe MANY publications that address this from the "make it testable from the start" point of view - virtually every publication of "extreme programming" or "test driven development" takes this approach.<BR/><BR/>On the other hand, when we wonder about code that was not written with tests in mind then I also perceive a dearth. There is to my knowledge one really good book on this exact issue though:<BR/><BR/>Michael Feathers' "Working Effectively with Legacy Code" - in fact, he defines "Legacy Code" not by age but as any code without good test coverage. The book provides many very practical techniques for making code testable and I recommend the book very much.<BR/><BR/><A HREF="http://my.safaribooksonline.com/0131177052" REL="nofollow">Safari - Read Online</A><BR/><BR/>or<BR/><BR/><A HREF="http://www.informit.com/store/product.aspx?isbn=9780131177055" REL="nofollow">http://www.informit.com/store/product.aspx?isbn=9780131177055</A>Matisse Enzerhttps://www.blogger.com/profile/03736762585596345292noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-40156575021712052512009-01-12T07:39:00.000-08:002009-01-12T07:39:00.000-08:00One more thing for Doug, who asked the original qu...One more thing for Doug, who asked the original question: here are a couple of references to testability that you might find useful. One is a PDF on <A HREF="http://www.satisfice.com" REL="nofollow">James Bach's Web site</A>, called <A HREF="www.satisfice.com/tools/testable.pdf" REL="nofollow"><I>Heuristics of Software Testability</I></A>. Another, mostly compiled by me, is on Adam Goucher's blog, <A HREF="http://adam.goucher.ca/?p=141" REL="nofollow">here</A>. This latter list suffers from a couple of omissions, the most serious of which Jerry pointed out in the SHAPE Forum (may it rest in peace): One of the most important dimensions of testability is, paradoxically, bug-free code. In a given unit of time, we can accomplish much more testing if we're not spending time investigating, recording, and reporting bugs that we find. Thus when a programmer carefully tests her code before giving it to someone to test, the tester can obtain different and presumably broader and deeper test coverage.<BR/><BR/>Maybe I'll write that book. :)<BR/><BR/>---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-34093687955689852332009-01-12T07:21:00.000-08:002009-01-12T07:21:00.000-08:00But you see, this is a type of decision that can't...<I>But you see, this is a type of decision that can't easily be caught by most kinds of testing. It is testing the design, I think, not the code. To do this kind of testing, the testers have to understand how the product will be used.</I><BR/><BR/>Implicit in this is the idea that most kinds of testing aren't useful to the people who are actually using the product. I hope we can change that. How do we do it—other than tying them to a chair and reading <A HREF="http://www.dorsethouse.com/books/perf.html" REL="nofollow"><I>Perfect Software and Other Illusions About Testing</I></A> or <A HREF="http://www.amazon.com/Lessons-Learned-Software-Testing-Kaner/dp/0471081124" REL="nofollow"><I>Lessons Learned in Software Testing</I></A> to them? :)<BR/><BR/>---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-52033359774371522682009-01-10T21:54:00.000-08:002009-01-10T21:54:00.000-08:00Michael wrote: "By the way, speaking of testing--d...Michael wrote: "By the way, speaking of testing--does it bother anyone else that Blogger shows the time of a posting, but not the date? "<BR/><BR/>Bothers me. It did right from the beginning. I've tried to imagine what could have possessed someone to think this is all right. Maybe they don't want you to realize that a post is three years old with zero responses.<BR/><BR/>But you see, this is a type of decision that can't easily be caught by most kinds of testing. It is testing the design, I think, not the code. To do this kind of testing, the testers have to understand how the product will be used.Gerald M. Weinberghttps://www.blogger.com/profile/05902673055244863609noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-44966793799617754522009-01-10T21:28:00.000-08:002009-01-10T21:28:00.000-08:00By the way, speaking of testing--does it bother an...By the way, speaking of testing--does it bother anyone else that Blogger shows the time of a posting, but not the date? That seems to apply both for blog posts and for comments.<BR/><BR/>---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-63529113008872360932009-01-10T21:27:00.000-08:002009-01-10T21:27:00.000-08:00Hi, Brian...I like the sentiment, but I'm not sure...Hi, Brian...<BR/><BR/>I like the sentiment, but I'm not sure I agree entirely with the idea "that each module be specified *in English* before coding begins." I used to think that way too, but these days, I might be more inclined to say "that each module be specified *in some comprehensible form*", and not necessarily before coding begins, but perhaps in parallel with it. Why? First, English (or the local vernacular) will certainly be important, but a working prototype, a sketch, a table, or a diagram might be more less expensive, quicker to produce, and more comprehensible than a written narrative. The key, to me, is in the ability for it to be reviewable by the people who matter--<I>and to make sure that they do review it</I>.<BR/><BR/>Second, it might be unreasonable and restrictive to expect that we can get it all right the first time, so cycles of experiments and learning might be a more reasonable approach. I think in the end, we certainly want to be able to <I>describe inputs, computations, validity tests, error conditions and messages, and finally outputs</I>. An important function of testing might be to aid in discovering and reporting that information, not merely in confirming it to be correct.<BR/><BR/><I>You can't test when you don't know what to expect.</I><BR/><BR/>That might be true to some degree--but Jerry has helped me to learn that sometimes you don't know what to expect until you've tested.<BR/><BR/>Cheers,<BR/><BR/>---Michael B.Michael Bolton http://www.developsense.comhttps://www.blogger.com/profile/09027725699187903416noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-29637962705331927772009-01-06T12:25:00.000-08:002009-01-06T12:25:00.000-08:00Another important "testability" strategy would be ...Another important "testability" strategy would be to require that each module be specified *in English* before coding begins.<BR/><BR/>Describe inputs, computations, validity tests, error conditions and messages, and finally outputs.<BR/><BR/>I realize that such standards are the exception, rather that the rule. Which makes for expensive, buggy software and meaningless testing - you can't test when you don't know what to expect.Brianhttps://www.blogger.com/profile/16818265261392759199noreply@blogger.comtag:blogger.com,1999:blog-25922407.post-50993399866256371972009-01-06T04:25:00.000-08:002009-01-06T04:25:00.000-08:00"What would it cost us every time this failure app..."What would it cost us every time this failure appears in the field?"<BR/><BR/>Excellent question. I have used this thought for years in many different types of software. It never made any sense to me to spend thousands of person hours to fix an error that showed itself once a year and cost the user two minutes to restart the application.<BR/><BR/>Money isn't everything in life, but sometimes it is a good idea to look at the cost in money (and the cost in other things) when testing software.Unknownhttps://www.blogger.com/profile/02305697664289102933noreply@blogger.com