Habits for Agile Testing

In addition to her many other talents, my wife, Dani, trains dogs. Her speciality is "problem dogs"—which  really means "problem owners."

Rather than retrain the dogs, Dani retrains the owners, who then retrain the dogs. In that sense, her dog work is very much like my program testing work. When people come to me with "problem programs," I work on the people, not the programs. After all, it was the people who made the programs into problems.

Of all the questions dog owners ask Dani, the most frequent is, "When do we start training the puppy?" Unfortunately, few people ask me the corresponding question, "When is the right time to start testing a program?"

What is the right time to start training a puppy? As Dani explains, the question is meaningless because "you start training a puppy the moment you set eyes on it, whether you intend to or not. And the puppy starts training you at the same time."

The same is true of testing programs. You start testing a program the moment you start to think of the possibility of writing a program. Actually, it starts even earlier.

How can program testing start even before you start thinking of writing the program? Think about the dog owners. Their troubles arise from bad habits they've acquired over a lifetime, before they ever set eyes upon their new puppy.

Your good and bad habits will have more influence than any other factor in the success or failure of your program testing efforts. It is no accident that certain programmers and testers perform consistently better than others in the production of well-tested programs. Those who believe it is an accident will learn nothing about program testing until they convince themselves it is possible to learn how to do a better testing job.

What Does Genius Require?

I'm not denying that a measure or basic intelligence is required to be a successful program tester. Even some forms of "genius" might help. But those who subscribe to the "native genius" school of programming should study the lives of geniuses.

For example, Einstein once remarked that he really didn't do anything but take the work of others and put it together in some way that had not been done before. If we are to emulate Einstein and put together the work of others—instead of starting from scratch with each new program—perhaps we can approach the genius level in our testing.

Scientists know that there is honor, not shame, in taking the work of others—probably smarter than ourselves—as the starting point for our work. For the most part, this previous work is available to us in libraries of one form or another.

Libraries may store millions of useful programming ideas, but their very size can make it difficult to know what they contain. In order to make effective use of libraries in program testing, we have to know what kinds of libraries are available, and we have to know what those libraries contain.

On library that every programmer uses is the one contained in the programming language. Even if your program is in binary machine code, you're using a library—the one the machine designers thought useful for programming. It is theoretically possible to design a general purpose computer with but a single instruction—a kind of "conditional branch and subtract." Therefore, any instruction set bigger than that must be considered a library—a way of giving the programmer a set of pre-tested building blocks.

By "higher-level" language, we essentially mean a language with a library that allows programs to be built with fewer building blocks. Fewer building blocks means fewer connections between building blocks—which in turn means fewer places to search for errors. Unfortunately, many programmers are not well acquainted with the contents of their language library.

Here's just one example. A programmer came to me with a program that was mysteriously losing parts of strings. I helped him narrow down the problem to a statement that was trying to replace two characters with asterisks, but when the original string was longer than 10 characters, the excess portion was truncated. When I asked him why he had used the "magic number" 10, he explained that at the time he wrote the statement he didn't think he would have strings longer than that.

He didn't know his language library contained contained the ability to do substring assignment. Had he known this, he could have written a much simpler and more reliable statement. That approach—because it was a more direct expression of what he wanted to do rather than how he wants it done—would have prevented the bug and other bugs from ever occuring in the first place.

Another genius, Newton, said he could see so far because "I am a midget standing on the shoulders of giants." Dick Hamming once told me the programmers are more like midgets standing on the toes of other midgets. It may be true that our language designers lack the mental stature of a Newton, but that's no reason to stand on their toes by ignoring the hidden libraries they have created for us. You can still see farther standing on shoulders than on toes.

Genius Libraries

But how can you come to know the contents of your hidden library? The obvious answer is to read the manual. Ugh! We complain that manuals are badly written, and many are. But a more serious problem is that manuals are written for reference, not to be read like novels. If you know there is such a thing as substring assignment, and you know its name, you can look it up and learn how it works. But if you don't suspect such a function exists, how can you find it in the manual?

Another problem with manual reading is their emphasis on how things work, rather than why you might want to use them in the first place. They are solution-oriented, rather than problem-oriented, leaving their readers to bridge the gap between what they're trying to accomplish and how the language could make it easier and more accurate.

Still, there are ways of using manuals to overcome some of this shortcomings. Two or more programmers can play a game. One takes the manual and reads the name of some feature to the others, who must then describe how that feature works. "What happens when we execute this?" Everyone writes down an answer, after which the example is executed and the competitive members of the team can award points for correct answers.

This game increases everyone's familiarity with parts of the hidden library—or any application, but still leaves the "why" largely unanswered. The game can be extended to "why?" by asking each player to come up with a problem for which his feature is the best solution. For substring assignment, a problem might be "to insert asterisks in the third and fourth positions of a string," or "to replace a blank at position N with a comma."

The technical review is even more effective at vocabulary building. Technical reviews have the further advantage that managers who might be troubled to see programmers and testers "playing games" have little trouble accepting a technical review for its obvious function of finding errors. In my experience, however, the long-range error-prevention benefits of technical reviews far outweigh their short-range debugging efforts.

When several people sit down to review code, each person can quietly witness all the good techniques used in that code. They can also witness poor techniques, and if anyone in the group happens to know a better technique, all can learn it. Moreover they can learn these things without having to confess that they didn't know them before the review.

Another problem with manuals is that though they may be comprehensive, they tend to put the emphasis on what is most difficult to explain, rather than what is most useful to know. Because technical reviews are focussed on current work, the learning from reviews tend to be more relevant than what you can learn from playing games.

There are, of course, other types of program library besides the hidden library of the programming language. There are procedure libraries, subroutine libraries, libraries containing tables of common data, utility libraries, libraries of of parameters to direct those utilities in common tasks, macro libraries containing common code sequences, data dictionaries, and the inventory of existing applications—often a huge library of useful code for reuse.

The amount of useful, pre-tested material in such libraries is so overwhelming that programmers and testers sometimes forget about that other sort of library—the kind containing good old-fashioned books. A caution, though. Even if we could somehow guarantee perfect copies of the author's examples, we must remember that examples presented in a book are ordinarily designed—or should be—for maximum teaching effect, not for maximum generality or performance. Books are terrific ways to obtain new ideas of what be in other types of library, which is where you should look for reliable pieces of code and design.

The subject of early testing would be incomplete without the mention of one more type of library—the library of work habits you carry around in your head, or wherever it is that work habits are carried. For instance, have you ever noticed how pig-pen workers invariably spend more time debugging than those with tidier habits? I say this not as Mr. Clean, but as an old pig-pen programmer myself. It took me thirty years of effort to clean up my act reasonably well, but I've never completely eliminated slovenly habits of my youth. Still, little by little, I've eliminated slovenly, lazy practices that had previously wasted thousands of hours of precious time.

Seven Simple Habits

The Chinese say, "a journey of 10,000 miles starts with a single step," and even one bug prevented is a step in the right direction of more reliable software. So, here are a few of the pre-programmed habits I've noticed in effective program testers:

Keeping a Journal

Like all professional engineers, professional programmers and testers should keep a bound journal in which to record all those tiny but significant events that occur every day. (I prefer a bound journal to a digital record because it prevents me from revising history to make myself look better than I really was.)

Ideas, observations, events, successes, failures, puzzles—all are grist for the journal. You don't have to realize the importance of something at the time you write it down. Indeed, the puzzling things are the most important to record. Often, bugs are found only when they have managed to repeat themselves in a dozen ways, and only with a journal will you have all the necessary data in one place. Memory isn't sufficiently reliable. Scraps of paper don't stay in one place. Your digital device my be in the next county when you have an idea, so carry a journal.

Record Dates and Times

Someone once said, "Time is God's way of making sure everything doesn't happen at once." Effective testers use God's great invention to ensure they always know the order of past events. Put the date and time on everything you write down, everything you put in the computer, and everything the computer puts out. Record both the time of writing and the time of the event itself. It's an investment like insurance—it may seem a bit troublesome at the time, but when the accident occurs, you'll be glad you bought it. And even if there's never an accident, you've bought some peace of mind.

Standardizing Formats

When I write down the same thing in different ways, I always seem to have trouble later. Europeans write dates as day/month; Americans use month/day. Obviously, either method is workable, but used together they make a mess. After living in Europe, I came to realize that the European method was clearly more logical (for sorting order, at least), but that didn't mean I could adopt it when I returned to the US. It confuses Americans. Instead, I evolved a standard (for me) dating method—11 October 2017) which is far less likely to confuse either Europeans or Americans than 10/11, or is it 11/10?

My standard method also forces me to write the year. You'd be surprised how many bugs I've had that survived more than a year—or maybe you wouldn't be surprised at all.

Over long periods of time, my ideas about certain standards have changed, which some people use as an argument against any kind of standard. But one advantage of standard formats is that they allow easier transformation to a new standard, should that be necessary. And, if I've dated everything and recorded the change in standards in my journal, I can always figure out which standard applied to a particular item.

Filing

One of the areas in which I still could use some improvement is my filing. My journal often helps as an index to filed items, and dating every filed item was a great leap forward. But I still find myself too frequently in the situation where I know I've put away the solution to a problem but can't find it in my files.

Clearly, I'm in no position to tell other people how to file their myriad documents—test results, bug reports, articles, correction letters, and so forth—that might be needed when testing a system. But at least I speak with the experience of a common person rather than a filing genius (like a former secretary who filed the cans of food in her kitchen alphabetically).

Probably the most important principle I've learned is to think, when putting something away, about the question I'll be asking when I want to retrieve it. That question is my best guide as to where to file it. Recently, when I've been unable to reduce an important item to a single question, I've taken to filing a reference to the item in extra places. Or, if it's filed digitally, I add at least one keyword for each possible question.

Throwing Things Away

Filing multiple references tends to compound another problem of mine, based on the universal law: "Files Fill." I've noticed that good testers have a sense of when to rid themselves of things. I don't have this sense, and it has always cost me much wasted effort when searching for something.

When I used to work for IBM (which stood for "I've Been Moved"), every time I packed my files I would purge about 1/3 of my filed material. Now that I don't move as frequently, I tend to drown in my files. Once in a while, though, I become seized with a cleaning fit and go through a drawer or two, or tidy up a few folders. Still, I'm still trying to learn something good about filing—or, rather, unfiling.

Learn Something New Before Sleeping

My father was an extraordinarily smart man. Somewhere in my early years, he told me that he would never go to bed at night unless he had learned something new that day. Somehow that made sense to me, perhaps because I wanted to be smart like my father. For whatever reason, I have followed this habit for my entire life. I was never one of those people who felt he could learn only in a classroom, so every night, before hitting the pillows, I review my day for learnings. My journal helps. If I cannot think of anything worth writing down, I go to my library and grab a book. Or, nowadays, I'll go online and browse until I find something of value.

Sometimes, it's something new. Others, it's a reminder of something I used to know, but have forgotten. Once in a great while, I try to retire without this learning, but I've discovered that this habit is so ingrained that I cannot fall asleep. So, I get up from bed and start the learning process over again. 

Learning From Others

Which brings me to the final habit on my list, undoubtedly the one that has done more than all the others to make me a better tester.

I don't know how I acquired this habit—I certainly didn't have it when I was in school. Somewhere along the line, I developed the habit of listening to other people. And watching them.

Of course, I try to learn from my own mistakes, my ego makes it hard to learn from them. Besides there are so many other people making mistakes that I can learn even more from their blunders. I think that's how Dani learned how to train dogs—and dog owners.

Managing Teams Congruently

In these times of an Agile revolution in software, development managers have been hard taxed to keep up with the necessary changes in the way they must do their jobs to be successful. Whatever methods they used previously often fail to work when applied to the problems presented by Agile teams (or other teams, for that matter). To help them out, I've written the sixth volume in my Quality Management Series on the subject of managing teams congruently. What follows is the opening of that book...

Part I. Achieving Congruent Management

Keep good natured ... no matter what you find ... you did not invent life, so do not hold yourself responsible for what life does to you; decline to lay blame on anyone. - Hugh Mearns

"Who was that masked man?" The townspeople never knew who the Lone Ranger was, but fans knew that his career as the masked defender of justice began in tragedy. He and his fellow Texas Rangers were ambushed by a gang of outlaws. He was the only one to escape with his life. He could have spent the rest of his life playing the victim, but instead he responded by building a new and more effective role for himself. The rest is (fictional) history—how the Wild West was tamed.

It's worth noting that the Lone Ranger possessed no secret weapon. The technology available to him was precisely what was available to the guys in the black hats. The difference was in the way he used his technology—and didn't use it. The typical bad guy had only one way to solve problems of control—namely, to gun somebody down. In remarkably few episodes did the Masked Man ever use his hardware, and then not to kill, but to disable the hardware of his opponent.

The Lone Ranger won the battle of Good over Evil because he had many behaviors at his disposal. He was clever, skilled with his hardware, persuasive, and, most of all, congruent. He never fired his gun in anger, or hatred, or revenge.

When the history of software engineering is written, the story will be the same: Thousands of individuals will respond to failure by playing victim, by pouring on hardware, or by attacking others. Only a few will respond congruently, creating new and more effective roles, and taming the Wild Profession.

Chapter 1. Curing the Addiction to Incongruence

An ounce of application is worth a ton of abstraction. - Booker T. Washington

Why do so many people respond incongruently in so many situations? For the most part, they continue to behave incongruently because they've become addicted to such behavior. So, if we're to improve the software professions, we'll need to know how to lessen incongruence—ideally, to cure it completely. This won't be easy.

Addictions are notoriously difficult to cure because most people don't recognize an addiction. Even if they do recognize it, they fail to understand its dynamic. (Figure A-1, below, shows a diagram of effects of the generalized addiction cycle.) These failures lead to the belief in several ineffective methods of curing the addiction. This chapter examines those failures and offers a practical method of cure.

Figure A-1. A diagram of effects of the generalized addiction cycle.

1.1 Force the Addict to Stop

The simplest idea about curing addiction to X is to stop the use of X, under the belief that X causes the addiction. X does not cause the addiction; the addiction dynamic causes the addiction. We know that's true because not everyone who is exposed to X becomes an addict. Take morphine addiction, for instance. Many people are exposed to morphine in hospitals and don't become addicted, because they have other ways of dealing with the various pains of their lives. Only those people who are inclined to believe (for personal, economic, cultural, or other reasons) that there is only one magical way to cure life's problems—only those people are likely to become morphine addicts.

The same is true, say, for the addiction to omitting various forms of testing (such as, technical reviews) to push the schedule. The only managers who become addicted to this practice are those who don't understand what the omitted testing would do for them, plus those managers who have no effective alternate ways of dealing with a schedule delay.

So, why not just forbid the skipping of planned tests? The biggest reason why not: there's absolutely no evidence that prohibition works, and lots of evidence that it doesn't. Sometimes it doesn't work because it's impossible to enforce the prohibition, as the entire population of the United States saw when alcohol was prohibited.

As Figure 1-1 shows, attempts to prohibit X lead the addict to make greater and greater attempts to do X. If the prohibition is strong enough, then the addict may not succeed in doing X, but the strength of the belief in X is unchanged. The addict is still addicted, and as soon as a weak spot in the prohibition appears, the addict will do X again.

In the case where X is skipping of planned testing, the addict might try to make up the schedule by faking the records to show that skipped testing was not skipped at all. Skipping necessary testing always leads to buggier products, which eventually leads to punishing the addict for ineffective testing. Now the addict is in even more pain, and thus more tempted to speed up the next delivery by skipping more testing.

Or perhaps the addict manages to move to a new position and avoid responsibility for the buggy product. In that case, X "worked" for the addict, which strengthens the belief in X.

So, either the pain increases or the faith in the incongruent practice increases—or both. None of this looks like a "cure."

Figure 1-1. The simplest idea for stopping an addiction to X is prohibiting the use of X. This may lead to success or failure, but it always leads the addict to increasing efforts to do X because nothing is done about the long-range pain. If the prohibition works, the addiction may not grow stronger, but neither does it grow weaker. If it doesn't work, the addiction grows stronger, and new methods of evading the prohibition of X may now be learned.

Prohibition then cannot stop addiction, though at best it may stop new people from becoming addicts. Nevertheless, in a work situation, we may not care if the person or persons stay addicted as long as they cannot practice the behavior. For example, the prohibition of code-patching can be absolutely enforced with an appropriate configuration management system. When such a system is first introduced, programmers who are addicted to patching invariably try dozens of ingenious ways to beat the prohibition. If there's a crack, they will surely find it. If not, they'll eventually find that they may as well work within the system, albeit grudgingly. Their belief system remains unchanged, though, and given the slightest chance, they'll slip right back into the old behaviors.

Because the existing addicts will try harder and harder to beat the prohibition, the cost of this way of stopping the behavior is at least an increase in policing activity, with the accompanying deterioration in the general climate. Nevertheless, in circumstances like patch prevention with configuration management, the benefit can be worth the cost.

In other circumstances, however, it's simply not possible to obtain such absolute prevention. For instance, many software engineers are addicted to fiddling with software tools to the point where it interferes with their productive work. They are not addicted to tools, but to the overuse of tools. Because tools are a necessity for programming, there's no conceivable way management can stop these tool addicts from having access to their addiction.

At its worst, prohibition creates a class of people who profit from helping addicts beat the prohibition. This class of people then grows and recruits more addicts. A case in point is the tool vendors, who have a legitimate function, but who also may have no scruples about pandering to tool-based addictions. Why else would we find that 70 percent of purchased software tools are never used productively, yet few vendors will give their customers a refund for a returned tool?

What Else Can Be Done?

(The complete chapter is in the book. Subsequent topics include

• Punishment
• Rescue
• Co-dependency or Co-addiction
• A Successful Cure

plus many helpful hints and suggestions.)

Managing Teams Congruently can be found on Leanpub.com as a single volume.

It may also be found as part of the Bargain Bundle, Quality Software.