Sunday, July 26, 2015

Habits for Agile Testing

In addition to her many other talents, my wife, Dani, trains dogs. Her speciality is "problem dogs"—which  really means "problem owners."

Rather than retrain the dogs, Dani retrains the owners, who then retrain the dogs. In that sense, her dog work is very much like my program testing work. When people come to me with "problem programs," I work on the people, not the programs. After all, it was the people who made the programs into problems.

Of all the questions dog owners ask Dani, the most frequent is, "When do we start training the puppy?" Unfortunately, few people ask me the corresponding question, "When is the right time to start testing a program?"

What is the right time to start training a puppy? As Dani explains, the question is meaningless because "you start training a puppy the moment you set eyes on it, whether you intend to or not. And the puppy starts training you at the same time."

The same is true of testing programs. You start testing a program the moment you start to think of the possibility of writing a program. Actually, it starts even earlier.

How can program testing start even before you start thinking of writing the program? Think about the dog owners. Their troubles arise from bad habits they've acquired over a lifetime, before they ever set eyes upon their new puppy.

Your good and bad habits will have more influence than any other factor in the success or failure of your program testing efforts. It is no accident that certain programmers and testers perform consistently better than others in the production of well-tested programs. Those who believe it is an accident will learn nothing about program testing until they convince themselves it is possible to learn how to do a better testing job.

What Does Genius Require?

I'm not denying that a measure or basic intelligence is required to be a successful program tester. Even some forms of "genius" might help. But those who subscribe to the "native genius" school of programming should study the lives of geniuses.

For example, Einstein once remarked that he really didn't do anything but take the work of others and put it together in some way that had not been done before. If we are to emulate Einstein and put together the work of others—instead of starting from scratch with each new program—perhaps we can approach the genius level in our testing.

Scientists know that there is honor, not shame, in taking the work of others—probably smarter than ourselves—as the starting point for our work. For the most part, this previous work is available to us in libraries of one form or another.

Libraries may store millions of useful programming ideas, but their very size can make it difficult to know what they contain. In order to make effective use of libraries in program testing, we have to know what kinds of libraries are available, and we have to know what those libraries contain.

On library that every programmer uses is the one contained in the programming language. Even if your program is in binary machine code, you're using a library—the one the machine designers thought useful for programming. It is theoretically possible to design a general purpose computer with but a single instruction—a kind of "conditional branch and subtract." Therefore, any instruction set bigger than that must be considered a library—a way of giving the programmer a set of pre-tested building blocks.

By "higher-level" language, we essentially mean a language with a library that allows programs to be built with fewer building blocks. Fewer building blocks means fewer connections between building blocks—which in turn means fewer places to search for errors. Unfortunately, many programmers are not well acquainted with the contents of their language library.

Here's just one example. A programmer came to me with a program that was mysteriously losing parts of strings. I helped him narrow down the problem to a statement that was trying to replace two characters with asterisks, but when the original string was longer than 10 characters, the excess portion was truncated. When I asked him why he had used the "magic number" 10, he explained that at the time he wrote the statement he didn't think he would have strings longer than that.

He didn't know his language library contained contained the ability to do substring assignment. Had he known this, he could have written a much simpler and more reliable statement. That approach—because it was a more direct expression of what he wanted to do rather than how he wants it done—would have prevented the bug and other bugs from ever occuring in the first place.

Another genius, Newton, said he could see so far because "I am a midget standing on the shoulders of giants." Dick Hamming once told me the programmers are more like midgets standing on the toes of other midgets. It may be true that our language designers lack the mental stature of a Newton, but that's no reason to stand on their toes by ignoring the hidden libraries they have created for us. You can still see farther standing on shoulders than on toes.

Genius Libraries

But how can you come to know the contents of your hidden library? The obvious answer is to read the manual. Ugh! We complain that manuals are badly written, and many are. But a more serious problem is that manuals are written for reference, not to be read like novels. If you know there is such a thing as substring assignment, and you know its name, you can look it up and learn how it works. But if you don't suspect such a function exists, how can you find it in the manual?

Another problem with manual reading is their emphasis on how things work, rather than why you might want to use them in the first place. They are solution-oriented, rather than problem-oriented, leaving their readers to bridge the gap between what they're trying to accomplish and how the language could make it easier and more accurate.

Still, there are ways of using manuals to overcome some of this shortcomings. Two or more programmers can play a game. One takes the manual and reads the name of some feature to the others, who must then describe how that feature works. "What happens when we execute this?" Everyone writes down an answer, after which the example is executed and the competitive members of the team can award points for correct answers.

This game increases everyone's familiarity with parts of the hidden library—or any application, but still leaves the "why" largely unanswered. The game can be extended to "why?" by asking each player to come up with a problem for which his feature is the best solution. For substring assignment, a problem might be "to insert asterisks in the third and fourth positions of a string," or "to replace a blank at position N with a comma."

The technical review is even more effective at vocabulary building. Technical reviews have the further advantage that managers who might be troubled to see programmers and testers "playing games" have little trouble accepting a technical review for its obvious function of finding errors. In my experience, however, the long-range error-prevention benefits of technical reviews far outweigh their short-range debugging efforts.

When several people sit down to review code, each person can quietly witness all the good techniques used in that code. They can also witness poor techniques, and if anyone in the group happens to know a better technique, all can learn it. Moreover they can learn these things without having to confess that they didn't know them before the review.

Another problem with manuals is that though they may be comprehensive, they tend to put the emphasis on what is most difficult to explain, rather than what is most useful to know. Because technical reviews are focussed on current work, the learning from reviews tend to be more relevant than what you can learn from playing games.

There are, of course, other types of program library besides the hidden library of the programming language. There are procedure libraries, subroutine libraries, libraries containing tables of common data, utility libraries, libraries of of parameters to direct those utilities in common tasks, macro libraries containing common code sequences, data dictionaries, and the inventory of existing applications—often a huge library of useful code for reuse.

The amount of useful, pre-tested material in such libraries is so overwhelming that programmers and testers sometimes forget about that other sort of library—the kind containing good old-fashioned books. A caution, though. Even if we could somehow guarantee perfect copies of the author's examples, we must remember that examples presented in a book are ordinarily designed—or should be—for maximum teaching effect, not for maximum generality or performance. Books are terrific ways to obtain new ideas of what be in other types of library, which is where you should look for reliable pieces of code and design.

The subject of early testing would be incomplete without the mention of one more type of library—the library of work habits you carry around in your head, or wherever it is that work habits are carried. For instance, have you ever noticed how pig-pen workers invariably spend more time debugging than those with tidier habits? I say this not as Mr. Clean, but as an old pig-pen programmer myself. It took me thirty years of effort to clean up my act reasonably well, but I've never completely eliminated slovenly habits of my youth. Still, little by little, I've eliminated slovenly, lazy practices that had previously wasted thousands of hours of precious time.

Seven Simple Habits

The Chinese say, "a journey of 10,000 miles starts with a single step," and even one bug prevented is a step in the right direction of more reliable software. So, here are a few of the pre-programmed habits I've noticed in effective program testers:

Keeping a Journal
Like all professional engineers, professional programmers and testers should keep a bound journal in which to record all those tiny but significant events that occur every day. (I prefer a bound journal to a digital record because it prevents me from revising history to make myself look better than I really was.)

Ideas, observations, events, successes, failures, puzzles—all are grist for the journal. You don't have to realize the importance of something at the time you write it down. Indeed, the puzzling things are the most important to record. Often, bugs are found only when they have managed to repeat themselves in a dozen ways, and only with a journal will you have all the necessary data in one place. Memory isn't sufficiently reliable. Scraps of paper don't stay in one place. Your digital device my be in the next county when you have an idea, so carry a journal.

Record Dates and Times
Someone once said, "Time is God's way of making sure everything doesn't happen at once." Effective testers use God's great invention to ensure they always know the order of past events. Put the date and time on everything you write down, everything you put in the computer, and everything the computer puts out. Record both the time of writing and the time of the event itself. It's an investment like insurance—it may seem a bit troublesome at the time, but when the accident occurs, you'll be glad you bought it. And even if there's never an accident, you've bought some peace of mind.

Standardizing Formats
When I write down the same thing in different ways, I always seem to have trouble later. Europeans write dates as day/month; Americans use month/day. Obviously, either method is workable, but used together they make a mess. After living in Europe, I came to realize that the European method was clearly more logical (for sorting order, at least), but that didn't mean I could adopt it when I returned to the US. It confuses Americans. Instead, I evolved a standard (for me) dating method—11 October 2017) which is far less likely to confuse either Europeans or Americans than 10/11, or is it 11/10?

My standard method also forces me to write the year. You'd be surprised how many bugs I've had that survived more than a year—or maybe you wouldn't be surprised at all.

Over long periods of time, my ideas about certain standards have changed, which some people use as an argument against any kind of standard. But one advantage of standard formats is that they allow easier transformation to a new standard, should that be necessary. And, if I've dated everything and recorded the change in standards in my journal, I can always figure out which standard applied to a particular item.

One of the areas in which I still could use some improvement is my filing. My journal often helps as an index to filed items, and dating every filed item was a great leap forward. But I still find myself too frequently in the situation where I know I've put away the solution to a problem but can't find it in my files.

Clearly, I'm in no position to tell other people how to file their myriad documents—test results, bug reports, articles, correction letters, and so forth—that might be needed when testing a system. But at least I speak with the experience of a common person rather than a filing genius (like a former secretary who filed the cans of food in her kitchen alphabetically).

Probably the most important principle I've learned is to think, when putting something away, about the question I'll be asking when I want to retrieve it. That question is my best guide as to where to file it. Recently, when I've been unable to reduce an important item to a single question, I've taken to filing a reference to the item in extra places. Or, if it's filed digitally, I add at least one keyword for each possible question.

Throwing Things Away
Filing multiple references tends to compound another problem of mine, based on the universal law: "Files Fill." I've noticed that good testers have a sense of when to rid themselves of things. I don't have this sense, and it has always cost me much wasted effort when searching for something.

When I used to work for IBM (which stood for "I've Been Moved"), every time I packed my files I would purge about 1/3 of my filed material. Now that I don't move as frequently, I tend to drown in my files. Once in a while, though, I become seized with a cleaning fit and go through a drawer or two, or tidy up a few folders. Still, I'm still trying to learn something good about filing—or, rather, unfiling.

Learn Something New Before Sleeping
My father was an extraordinarily smart man. Somewhere in my early years, he told me that he would never go to bed at night unless he had learned something new that day. Somehow that made sense to me, perhaps because I wanted to be smart like my father. For whatever reason, I have followed this habit for my entire life. I was never one of those people who felt he could learn only in a classroom, so every night, before hitting the pillows, I review my day for learnings. My journal helps. If I cannot think of anything worth writing down, I go to my library and grab a book. Or, nowadays, I'll go online and browse until I find something of value.

Sometimes, it's something new. Others, it's a reminder of something I used to know, but have forgotten. Once in a great while, I try to retire without this learning, but I've discovered that this habit is so ingrained that I cannot fall asleep. So, I get up from bed and start the learning process over again. 

Learning From Others
Which brings me to the final habit on my list, undoubtedly the one that has done more than all the others to make me a better tester.

I don't know how I acquired this habit—I certainly didn't have it when I was in school. Somewhere along the line, I developed the habit of listening to other people. And watching them.

Of course, I try to learn from my own mistakes, my ego makes it hard to learn from them. Besides there are so many other people making mistakes that I can learn even more from their blunders. I think that's how Dani learned how to train dogs—and dog owners.

Wednesday, July 01, 2015

Managing Teams Congruently

In these times of an Agile revolution in software, development managers have been hard taxed to keep up with the necessary changes in the way they must do their jobs to be successful. Whatever methods they used previously often fail to work when applied to the problems presented by Agile teams (or other teams, for that matter). To help them out, I've written the sixth volume in my Quality Management Series on the subject of managing teams congruently. What follows is the opening of that book...

Part I. Achieving Congruent Management

Keep good natured ... no matter what you find ... you did not invent life, so do not hold yourself responsible for what life does to you; decline to lay blame on anyone. - Hugh Mearns

"Who was that masked man?" The townspeople never knew who the Lone Ranger was, but fans knew that his career as the masked defender of justice began in tragedy. He and his fellow Texas Rangers were ambushed by a gang of outlaws. He was the only one to escape with his life. He could have spent the rest of his life playing the victim, but instead he responded by building a new and more effective role for himself. The rest is (fictional) history—how the Wild West was tamed.

It's worth noting that the Lone Ranger possessed no secret weapon. The technology available to him was precisely what was available to the guys in the black hats. The difference was in the way he used his technology—and didn't use it. The typical bad guy had only one way to solve problems of control—namely, to gun somebody down. In remarkably few episodes did the Masked Man ever use his hardware, and then not to kill, but to disable the hardware of his opponent.

The Lone Ranger won the battle of Good over Evil because he had many behaviors at his disposal. He was clever, skilled with his hardware, persuasive, and, most of all, congruent. He never fired his gun in anger, or hatred, or revenge.

When the history of software engineering is written, the story will be the same: Thousands of individuals will respond to failure by playing victim, by pouring on hardware, or by attacking others. Only a few will respond congruently, creating new and more effective roles, and taming the Wild Profession.

Chapter 1. Curing the Addiction to Incongruence

An ounce of application is worth a ton of abstraction. - Booker T. Washington

Why do so many people respond incongruently in so many situations? For the most part, they continue to behave incongruently because they've become addicted to such behavior. So, if we're to improve the software professions, we'll need to know how to lessen incongruence—ideally, to cure it completely. This won't be easy.

Addictions are notoriously difficult to cure because most people don't recognize an addiction. Even if they do recognize it, they fail to understand its dynamic. (Figure A-1, below, shows a diagram of effects of the generalized addiction cycle.) These failures lead to the belief in several ineffective methods of curing the addiction. This chapter examines those failures and offers a practical method of cure.
Figure A-1. A diagram of effects of the generalized addiction cycle.

1.1 Force the Addict to Stop

The simplest idea about curing addiction to X is to stop the use of X, under the belief that X causes the addiction. X does not cause the addiction; the addiction dynamic causes the addiction. We know that's true because not everyone who is exposed to X becomes an addict. Take morphine addiction, for instance. Many people are exposed to morphine in hospitals and don't become addicted, because they have other ways of dealing with the various pains of their lives. Only those people who are inclined to believe (for personal, economic, cultural, or other reasons) that there is only one magical way to cure life's problems—only those people are likely to become morphine addicts.

The same is true, say, for the addiction to omitting various forms of testing (such as, technical reviews) to push the schedule. The only managers who become addicted to this practice are those who don't understand what the omitted testing would do for them, plus those managers who have no effective alternate ways of dealing with a schedule delay.

So, why not just forbid the skipping of planned tests? The biggest reason why not: there's absolutely no evidence that prohibition works, and lots of evidence that it doesn't. Sometimes it doesn't work because it's impossible to enforce the prohibition, as the entire population of the United States saw when alcohol was prohibited.

As Figure 1-1 shows, attempts to prohibit X lead the addict to make greater and greater attempts to do X. If the prohibition is strong enough, then the addict may not succeed in doing X, but the strength of the belief in X is unchanged. The addict is still addicted, and as soon as a weak spot in the prohibition appears, the addict will do X again.

In the case where X is skipping of planned testing, the addict might try to make up the schedule by faking the records to show that skipped testing was not skipped at all. Skipping necessary testing always leads to buggier products, which eventually leads to punishing the addict for ineffective testing. Now the addict is in even more pain, and thus more tempted to speed up the next delivery by skipping more testing.

Or perhaps the addict manages to move to a new position and avoid responsibility for the buggy product. In that case, X "worked" for the addict, which strengthens the belief in X.

So, either the pain increases or the faith in the incongruent practice increases—or both. None of this looks like a "cure."

Figure 1-1. The simplest idea for stopping an addiction to X is prohibiting the use of X. This may lead to success or failure, but it always leads the addict to increasing efforts to do X because nothing is done about the long-range pain. If the prohibition works, the addiction may not grow stronger, but neither does it grow weaker. If it doesn't work, the addiction grows stronger, and new methods of evading the prohibition of X may now be learned.

Prohibition then cannot stop addiction, though at best it may stop new people from becoming addicts. Nevertheless, in a work situation, we may not care if the person or persons stay addicted as long as they cannot practice the behavior. For example, the prohibition of code-patching can be absolutely enforced with an appropriate configuration management system. When such a system is first introduced, programmers who are addicted to patching invariably try dozens of ingenious ways to beat the prohibition. If there's a crack, they will surely find it. If not, they'll eventually find that they may as well work within the system, albeit grudgingly. Their belief system remains unchanged, though, and given the slightest chance, they'll slip right back into the old behaviors.

Because the existing addicts will try harder and harder to beat the prohibition, the cost of this way of stopping the behavior is at least an increase in policing activity, with the accompanying deterioration in the general climate. Nevertheless, in circumstances like patch prevention with configuration management, the benefit can be worth the cost.

In other circumstances, however, it's simply not possible to obtain such absolute prevention. For instance, many software engineers are addicted to fiddling with software tools to the point where it interferes with their productive work. They are not addicted to tools, but to the overuse of tools. Because tools are a necessity for programming, there's no conceivable way management can stop these tool addicts from having access to their addiction.

At its worst, prohibition creates a class of people who profit from helping addicts beat the prohibition. This class of people then grows and recruits more addicts. A case in point is the tool vendors, who have a legitimate function, but who also may have no scruples about pandering to tool-based addictions. Why else would we find that 70 percent of purchased software tools are never used productively, yet few vendors will give their customers a refund for a returned tool?

What Else Can Be Done?

(The complete chapter is in the book. Subsequent topics include
  • Punishment
  • Rescue
  • Co-dependency or Co-addiction
  • A Successful Cure

plus many helpful hints and suggestions.)

Managing Teams Congruently can be found on as a single volume.

It may also be found as part of the Bargain Bundle, Quality Software.

Tuesday, May 05, 2015

Requirements Hints and Variations

In both volumes of Exploring Requirements, we follow each chapter with a section of hints and variations about the topic of the chapter. Many readers tell us that this extra material is often more valuable than the chapter body itself. For that reason, I've decided to present a blog entry consisting entirely of  the hints and variations from the first chapter of Volume One. Enjoy!

(1) Another source of complexity in development projects is the desire of non-experts to be involved in the design of complex products. Most customers naturally prefer to learn as little as possible about product design, yet still have all their wishes met in the requirements process. When customers prefer to remain naive about the product design process, most of the burden falls on the notation—a lot to ask of a notation. The design complexity problem can be solved by creating a tutorial to convert a naive customers into an expert—but not if the customers doesn’t want to invest that kind of effort.

(2) Customers often turn away from the design process because the professional designers treat them in a patronizing manner. Remember, most participants are naive only about the development process, and are themselves experts in subjects about which the designers are naive. Such expertise is why their participation is essential. We’re much more likely to have their full participation if we create an environment of shared expertise.

(3) Methodology experts underestimate the difficulty of understanding their nota- tions, which they believe to be “intuitive.” To understand how difficult it really is to define an “intuitive” notation, consider the problem of designing universally understood international road signs.

(4) When two sets of experts participate in the same requirements process, there’s often a conflict about which notation is intuitive. Children reared in Paris think French is intuitive, but children reared in Montreal may think both French and English are intuitive. In the same way, experts often share a “first love” syndrome for notations. This first-learned-first-loved bias can be prevented in the same way bilingual children avoid a bias for one language. If you are teaching a notation, instruct your students in two at the same time. Do all maps in both notations, and compare the pair of maps explicitly.

(5) Mastery of the notation may give one party dominance in the requirements process, while excluding those who haven’t mastered the notation. To avoid political wars, you must devote attention to depoliticizing the language issue. Try following the Swiss example: All “first love” notations of participants are declared to be “official” languages of the requirements process. Every official document must be presented in all of the official languages before the process is considered complete.

(6) Although this multilingual approach may seem burdensome, the Swiss example shows it can work if done in the right spirit. Indeed, in Swiss meetings, people from the “dominant” language group often present their ideas in the “secondary” languages, as a courtesy to the minority participants and as a strong indication of how much they value their participation. In our experience with requirements work, this multilingual approach always yields a more accurate definition of requirements as well as a more complete involvement of all participants.

(7) Our colleague Eric Minch suggests a theoretical model: all descriptions of the de- signed system—including the various constituencies’ requirements and satisfactions, the constraints and decisions, and the final specifications for implementation—can be considered statements in different “languages.” In other words, instead of seeing all of these as different descriptions in a single language, we can think of them as the same description in different languages. The full design task then involves finding a way of translating among these languages, and the final product is such a translation strategy.

(8) Minch’s idea stimulated us to recall “translation” is not quite the simple one-to-one task we often assume. Some translations are works of art in themselves, like Edward FitzGerald’s English translation of The Rubaiyat of Omar Khayyam. Consider the famous quatrain:

A Book of Verses underneath the Bough, 
A Jug of Wine, a Loaf of Bread—and
Thou Beside me Singing in the Wilderness—
Oh, Wilderness were Paradise enow!

What part is original with Omar the Tentmaker, astronomer and mathematician from eleventh century Persia? What part has been added (or subtracted) by FitzGerald, the nineteenth century Suffolk gentleman? What part has been added by twenty-first century readers, such as us?

(9) Experiencing the final product, we don’t so much care if it’s an exact translation, but only whether we like the product. This verse reminds us: value can be added at any stage of the product development cycle. Requirements are merely a guide. They are to be taken literally, but not too literally. There are many roads to Paradise.

(10) These observations connect directly with a remark by Ken de Lavigne, our colleague at IBM: “Your discussion of maps brings out the great problem introduced by ‘stepwise refinement’ approaches: although they claim to postpone decisions until the time comes to make them, in fact the most far-reaching decisions are made first, when one has the least amount of information.” Always be prepared to go back and revise the “translation,” or “map,” when further exploration shows there was a wrong branch higher in the decision tree. To do this, let go of the idea of the “one right way,” or “the perfect translation.”

Extra Hint
To see an annotated catalog of all of Jerry's ebooks and bargain bundles, visit

To see a list of most of his books, ebooks and bound books, visit

Sunday, April 26, 2015

Ending the Requirements Process

This essay is the entire chapter 5 of the second volume of Exploring Requirements.
The book can be purchased as a single volume, or volumes 1 and 2 as a bundle, or as part of the People Skills bundle. It's also available from various vendors with both volumes as one bound book.

The Book of Life begins with a man and woman in a garden. It ends with Revelations.—Oscar Wilde, A Woman of No Importance
25.1 The Fear of Ending
The requirements process begins with ambiguity. It ends not with revelations, but with agreement. More important, it ends.
But how does it end? At times, the requirements process seems like Oscar Wilde when he remarked, "I was working on the proof of one of my poems all the morning and took out a comma. In the afternoon, I put it back again."
A certain percentage, far too large, of development efforts never emerge from the requirements phase. After two, or five, or even ten years, you can dip into the ongoing requirements process and watch them take out a comma in the morning and put it back again in the afternoon. Far better the comma should be killed during requirements than allowed to live such a lingering death.
Paradoxically, it is the attempt to finish the requirements work that creates this endless living death. The requirements phase ends with agreement, but the requirements work never ends until the product is finished. There simply comes a moment when you decide you have enough agreement to risk moving on into the full design phase.
25.2 The Courage to End It All
Nobody can tell you just when to step off the cliff. It's simply a matter of courage. Whenever an act requires courage, you can be sure that people will invent ways of reducing the courage needed. The requirements process is no different, and several inventions are available to diminish the courage needed to end it all.
25.2.1 Automatic design and development
One of the persistent "inventions" to substitute for courage is some form of automatic design and/or development.
In automatic development, the finished requirements are the input to an automatic process, the output of which is the finished product. There are, today, a few simple products that can be produced in approximately this way. For instance, certain optical lenses can be manufactured automatically starting from a statement of requirements.
In terms of the decision tree, automatic development is like a tree with a trunk, one limb, no branches or twigs, and a single leaf (Figure 25-1). With such a system, finishing requirements is not a problem. When you think you might be finished, you press the button and take a look at the product that emerges. If it's not right, then you weren't finished with the requirements process.
Figure 25-1. Automatic development is like a tree with a trunk, one limb, no branches or twigs, and a single leaf.
No wonder this appealing dream keeps recurring. It's exactly like the age-old story of the genie in the bottle, with no limit on the number of wishes. For those few products where such a process is in place, the only advantage of a careful requirements process is to save the time and money of wasted trial products. If trial products are cheap, then we can be quite casual about finishing requirements work.
25.2.2 Hacking
Automatic development is, in effect, nothing but requirements work—all trunk and limb. At the other end of the spectrum is hacking, development work with no explicit requirements work. When we hack, we build something, try it out, modify it, and try it again. When we like what we have, we stop. In terms of the decision tree model, hacking is not a tree at all, but a bush: all branches, twigs, and leaves, with no trunk or major limb (Figure 25-2).
Figure 25-2. Hacking is not a tree at all, but a bush—all branches, twigs, and leaves, with no trunk or major limb.
Pure hacking eliminates the problem of ending requirements work, because there is no requirements work. On the other hand, we could conceive of hacking as pure requirements work—each hack is a way of finding out what we really want.
Almost any real project, no matter how well planned and managed, contains a certain amount of hacking, because the real world always plays tricks on our assumptions. People who abhor the idea of hacking may try to create a perfect requirements process. They are the very people who create "living death" requirements processes.
25.2.3 Freezing requirements
Paradoxically, hacking and automatic development are exactly the same process from the point of view of requirements. They are the same because they do not distinguish requirements work from development. Most real-world product development falls somewhere between pure hacking and automatic development, which is why we have to wrestle with the problem of ending.
Even when we have every requirement written down in the form of an agreement, we cannot consider the requirements process to be ended. We know those agreements will have to change because in the real world, assumptions will change. Some people have tried to combat their fear of changing assumptions by imposing a freeze on requirements. They move into the design phase with the brave declaration,
No changes to requirements will be allowed.
Those of us who understand the nature of the real world will readily understand why a freeze simply cannot work. We know of only one product development when it was even possible to enforce the freeze. In that instance, a software services company took a contract to develop an inventory application for a manufacturer. The requirements were frozen by being made part of the contract, and eighteen months later, the application was delivered. Although it met all the contracted requirements, the application was rejected.
This frozen product was totally unusable because in eighteen months it had become totally removed from what was really required in the here-and-now. The customer refused to pay, and the software services company threatened legal action. The customer pointed out how embarrassing it would be to a professional software company to have its "freeze fantasy" exposed in a public courtroom.
Eventually, the two parties sat down to negotiate, and the customer paid about one-fourth of the software firm's expenses. At the end of this bellicose negotiation, someone pointed out how with far less time negotiating, they could have renegotiated the requirements as they went along, and both parties would have been happy.
25.2.4 The renegotiation process
The freeze idea is just a fantasy, designed to help us cope with our fear of closure. But we cannot fearlessly close the requirements phase unless we know there is some renegotiation process available. That's why agreeing to a renegotiation process is the last step in the requirements process.
Working out the renegotiation process is also a final test of the requirements process itself. If the requirements process has been well done, the foundation has been laid for the renegotiation process. The people have all been identified, and they know how to work well together. They have mastered all the techniques they will need in renegotiation, because they are the same techniques the participants needed to reach agreement in the first place.
25.2.5 The fear of making assumptions explicit
The agreement about renegotiation must, of course, be written down, and this act itself may strike fear in some hearts. Another way to avoid ending requirements is to avoid written agreements at the end of the requirements phase. Some designers are afraid of ending requirements because the explicit agreements would make certain assumptions explicit. An example may serve to make this surprising observation more understandable.
While working with the highway department of a certain state, we encountered the problem of what to do about a particularly dangerous curve on one of the state highways (see Figure 25-3). In an average year, about six motorists missed the curve and went to their death over a cliff. Because it was a scenic highway, it was neither practical nor desirable to eliminate the curve, but highway design principles indicated that a much heavier barrier would prevent wayward cars from going over the cliff.
Figure 25-3. What should be done about this dangerous curve?
Building the barrier seems an obvious decision, but there was another factor to consider. Perhaps once every three years, with a heavy barrier in place, one of these wayward cars would bounce off the barrier into a head-on collision with an oncoming vehicle. The collision would likely be fatal for all involved.
Now, on average, the number of people killed with the barrier in place would be perhaps one-fifth of those killed without the barrier, but the highway designers had to think of another factor. When a solo driver goes over the cliff, the newspapers will probably blame the driver. But what if a drunk driver bounces off the barrier and kills a family of seven who just happened to be driving in the wrong place at the wrong time? The headlines would shout about how the barrier caused the deaths of innocent people, and the editorials would scream for heads to roll at the highway department.
So, it's no wonder the highway engineers didn't want anything documented about their decision not to build the barrier. They were making life-and-death decisions in a way that covered their butts, and they could protect themselves by taking the position, "If I never thought about it, I'm not responsible for overlooking it in the design." And, if it was never written down, who could say they had thought of it?
25.3 The Courage to Be Inadequate
Most engineers and designers react to this story by citing similar stories of their own to prove, yes, indeed, there are decisions it's better not to write down. We believe, however, this kind of pretense is an abuse of professional power, an abuse not necessary if we remember the proper role of the requirements process.
It's not for the designers to decide what is wanted, but only to assist the customers in discovering what they want. The highway designers should have documented the two sides of the issue, then gone to the elected authorities for resolution of this open requirements question. With guidance from those charged with such responsibilities, the engineers could have designed an appropriate solution.
But suppose the politicians came back with an impossible requirement, such as,
The highway curve must be redesigned so there will be no fatalities in five years.
Then the engineers would simply go back to their customers and state they knew of no solution that fit this requirement, except perhaps for a barricade preventing cars from using the highway. Yes, they might lose their jobs, but that's what it means to be a professional—never to promise what you know in advance you can't deliver.

The purpose of requirements work is to avoid making mistakes, and to do a complete job. In the end, however, you can't avoid all mistakes, and you can't be omniscient. If you can't risk being wrong—if you can't risk being inadequate to the task you've taken on—you will never succeed in requirements work. If you want the reward, you will have to take the risk.

Tuesday, March 31, 2015

Three Samples to Read: People Skills—Soft but Difficult

Over more than half-a-century as a "technical consultant," I've learned that most of the problems I've seen are not fundamentally technical problems, but problems arising from people—problems solved or prevented by the application of so-called "soft" skills."
When we call these people-skills "soft," we imply that they are "easy" problems. Not so. If anything, they are hard problems—especially for those of us who have chosen some "technical" profession.

I've gradually developed some skill at handling such "soft" problems. I've also developed some skill at teaching others to improve their ability to solve such problems. "Teaching," of course, is another one of these "soft" skills. Writing is yet another "soft" (but not easy) skill, and I have written a number of books on a variety of "people skills."

My clients have often asked me to write one giant book covering "all the people skills," but such a book would have to have more that a thousand pages, so I have not done so. (One of my people skills is understanding that there are few people who would even attempt to read a thousand-page book.)

Recently, though, one of my clients pointed out that I had already written much more than a thousand pages on people skills (duh!). Once I let that "obvious" information into my thick skull, I realized that I knew a way to make some of those pages more readily available to my readers. And so I created a bundle of seven e-books anyone can buy at a saving of 25% of the original prices.

To help you decide if this bundle is for you, I've created the following sample of small lessons clipped from three of my books.  You can read and enjoy these samples and decide if your own "people skills" could be improved by reading dozens of such little lessons, plus a number of large lessons in the form of pragmatic models of human behavior. If your answer is "yes," then I invite you to purchase the bundle—risk-free, with Leanpub's "100% happiness guarantee."

So, here they are. Enjoy!

Sample #1: Discussing the Indiscussible 

(from More Secrets of Consulting)

Over the years I’ve come to believe that they key moment in an relationships occurs when one or both partners feels there’s something that can’t be talked about. This could be one thing, or many things, and for a lot of different reasons. When that moment arrives, the one thing that must happen is that the two partners talk about that thing. 

As a consultant, one of the most important things I can do to improve relationships is get that indiscussible subject out and on the table—but that seems risky. When I begin to fear this risk, I remind myself of the terrible consequences I’ve seen when the partners don’t discuss the taboo subject.

For instance, I was asked to work with Bill and Sherman, the co- developers of a software product who weren’t talking with one another. In this case, the thing that Bill and Sherman needed to talk about was that the Sherman didn’t want to talk to Bill. He didn’t even want to talk about the fact that he didn’t want to talk to Bill, so I decided to approach the subject indirectly, to reduce risk. I went to see Sherman and showed him my Courage Stick, letting him hold it and feel it’s smoothness. 

“It’s nice,” Sherman said. “What is it?” 

“It’s my Courage Stick. I brought it with me because I wanted to talk to you about something and I was a little afraid you might not like it.” 

“You, afraid of me? I’ve never seen you afraid to say anything.”
“Oh, I get scared about lots of things I need to say, but my Courage Stick reminds me that there are also fearful consequences when important things aren’t said.” 

“Like what?” 

“Like if I don’t tell you what will happen to your company if you and Bill don’t talk about some essential subjects. Like how you’re going to have a product that sucks, and how everybody is going to infer that Sherman is a lousy software architect.” 

You see, I didn’t know why Sherman was afraid to talk to Bill, but I knew that this would tap into one of Sherman’s greatest fears and change the Fraidycat Formula. I never tried to get Sherman to admit that he was afraid of talking with Bill, and after a little more coaxing, I led him by the elbow down to Bill’s office. I stayed for a while to act as referee, but soon Sherman’s fear was a thing of the past. Later, Bill told me that it took a lot of courage for Sherman to approach him. I didn’t bother to correct his impression. 

Sample #2: Changing Geography

(A case study from Becoming a Change Artist)

Change is a long-term process, but a living organization lives in the immediate present. Thus, without careful management, long-term change is invariably sacrificed to short-term expedience. And short-term expedience is taking place all the time, everywhere in the organization, essentially out of the view of high-level management. That's why change artists have to be in every nook and cranny, as the following case illustrates.

DeMarco and Lister have given us much useful information on the effects of geography on software development effectiveness. One manager, Ruben, reading Peopleware, was inspired to change the seating geography to improve customer relations. His semi-annual customer satisfaction survey had given the developers very low marks on communication. So, instead of seating people for efficient performance of today's work, he wanted to seat them to encourage communication, by putting eight developers in the customers' offices. However, when he surveyed customers six months later, Ruben found a large decline in their satisfaction with communication, precisely in those offices into which he had moved a developer.
What had typically happened was this. The developer would move to the customer's office and set up shop. Communication improved, but the first time there was a software emergency, the developer would rush back "home" to get the problem solved. Emergencies were rather frequent, and soon the developer would find it expedient to establish a "temporary" office in the developer area. 

After a while, the temporary office was occupied 99% of the time. In terms of the Satir Change Model, the foreign element—Ruben's move—had been rejected. Even worse, the customers were left staring at an empty office they were paying for, which reminded them of how hard it was to communicate with developers. 

Ruben noticed, however, that in one customer's office, satisfaction increased. In that office, the scenario had been different. Polly, a customer and change artist, sat in the office next door to Lyle, the developer, and noticed how often he was missing. She listened to comments made by other customers, and then she took action. 

Interviewing Lyle, she discovered that there were two main reasons why he kept leaving to solve problems: 

  • The PCs in the customer office had less capacity than the ones in the developers' office, and that capacity was needed to run debugging tools effectively. 
  • There were two other developers Lyle needed to consult on most of these problems, and they still resided in the developers' office. 
Polly knew these were typical problems of the Integration phase of the Satir Change Model, so she simply arranged to have Customer Service upgrade Lyle's PC. She then explained the benefits of having the two developers come to the customers' office, which was, after all, only two floors away. They were only too glad to come, and the customers were fascinated to learn how much work it was to fix "simple" problems in software.

With Polly as Lyle's neighbor, Ruben's strategic concept was implemented. Without using his survey to connect the strategic and the tactical, he would never have learned that it was possible to make the new geography work, and the success would have been limited to Lyle's area. Polly was sent around to the other developers who had moved, and she managed to get five out of seven working smoothly by similarly upgrading their PCs and encouraging developers to solve problems close to where they occurred. 

Polly also had the change-artist's skill to recognize that the remaining two departments were having deeper problems with information systems—problems that wouldn't be solved by upgrading PCs or encouraging proximity. Indeed, she saw that proximity was only making matters worse because the people were unskilled in handling interpersonal conflict. Thus, instead of blindly applying the same solution to everyone, she suggested to the managers that certain people could benefit from training in teamwork skills. 

Sample #3: Context-Free Questions

(from Exploring Requirements)

Once we have a starting point and a working title for a project, our next step in the project is invariably to ask some context-free questions. These are high-level questions posed early in a project to obtain information about global properties of the design problem and potential solutions.

Context-free questions are completely appropriate for any product to be designed, whether it's an automobile, airplane, ship, house, a jingle for a one-minute television commercial for chewing gum, a lifetime light bulb, or a trek in Nepal.

In terms of the decision tree, context-free questions help you decide which limb to climb, rather than which branch or twig. Because context-free questions are independent of the specific design task, they form a part of the designer's toolkit most useful before getting involved in too many enticing details.

Context-Free Process Questions
Some context-free questions relate to the process of design, rather than the designed product. Here are some typical context-free process questions, along with possible answers for the Elevator Information Device Project. (Notice that because these questions are context-free, we don't need to understand anything about the Elevator Information Device Project, an example started earlier in the Exploring Requirements book).

To appreciate how these questions can always be asked, regardless of the product being developed, also try answering them for some current project of yours, like "Trekking in Nepal."

Q: Who is the client for the Elevator Information Device?
A: The client is the Special Products Division of HAL, the world's largest imaginary corporation.

Q: What is a highly successful solution really worth to this client?
A: A highly successful solution to the problem as stated would be worth $10 million to $100 million in increased annual net profit for a period of five to ten years. The product should start earning revenue at this rate two years from now.

Q: (Ask the following context-free question if the answer to the previous context-free question does not seem to justify the effort involved.) What is the real reason for wanting to solve this problem?
A: The Elevator Information Device Project is a pilot effort for a range of commercial (and possibly even home or personal) information transfer devices. If we can demonstrate success during development and early marketing phases, this project is expected to spawn an independent business unit with gross revenues in seven years of $2 billion per year.

Q: Should we use a single design team, or more than one? Who should be on the team(s)?
A: You may choose whatever team structure you desire, but include someone on the team who knows conventional elevator technology, and someone who understands building management.

Q: How much time do we have for this project? What is the trade-off between time and value?
A: We don't need the device before two years from now because we won't be ready to market it, but every year we delay after that will probably reduce our market share by half.

Q: Where else can the solution to this design problem be obtained? Can we copy something that already exists?
A: To our knowledge, nowhere. Although many solutions exist in the form of control and information display panels for elevators, the approach adopted here should exploit the latest in sensing, control, information display, and processing—so copying doesn't seem appropriate. We have no objection to your copying features existing elsewhere, and you should be aware of what else has been done in the field, so you can surpass it.

Potential Impact of a Context-Free Question
Are context-free questions really worth such a fuss, and why do they need to be asked so early? Let's look at another example—extreme but real—of an answer to the second question, "What is a highly successful solution really worth to this client?"

At 3 a.m., a man in dirty jeans and cowboy boots showed up at the service bureau operation of a large computer manufacturer. Through the locked door, he asked if he could buy three hours' worth of computer time on their largest machine—that night. The night-shift employees were about to turn him away when one of them said, "Well, it costs $800 an hour. Is it worth $2,400?"

"Absolutely," said the cowboy, who emphasized the urgency by pulling a large wad. of $100 bills from his pocket and waving them at the employees on the other side of the glass door. They let him enter, took his payment in cash, and allowed him run his job on the machine. It turned out he owned a number of oil wells. As result of his computations, and especially the courteous treatment he received, he bought three of the giant machines, at a cost of some $10 million. If the employees had assumed the answer to the "What's it worth?" question based on his appearance, there would have been no sale.

The People-Skills Bundle

To learn more about all 7 books in the bundle, take a look at 

If you like what you see, buy the bundle with its 100% happiness guarantee. And perhaps you have a friend or two who could profit from reading their own copy.