Sunday, September 23, 2012

Agile and the Definition of Quality


Some Agile writers have called me "the grandfather of Agile." I choose to interpret that comment as a compliment, rather than a disparagment of my advanced age. As a grandfather, much of my most influential writing was done long before the Agile movement appeared on stage. As a result, newcomers on the scene often fail to see the connection between those writings and today's Agile movement.
I'm planning to use my blog to correct that situation, with a series of articles relating specific material to Agile basics. I'm starting with this blog entry about my definition of "quality"–often quoted by not always understood. The essay is adapted from the very first chapter of How Software is Built, which in turn is adapted my the first volume of Quality Software Management. <http://www.geraldmweinberg.com/Site/QSM_vol_1.html>



A Bug in the Family
My sister's daughter, Terra, is the only one in the family who has followed Uncle Jerry in the writer's trade. She writes fascinating books on the history of medicine, and I follow each one's progress as if it were one of my own. For that reason, I was terribly distressed when her first book, Disease in the Popular American Press, came out with a number of gross typographical errors in which whole segments of text disappeared. I was even more distressed to discover that those errors were caused by an error in the word processing software she used–CozyWrite, published by one of my clients, the MiniCozy Software Company.
Terra asked me to discuss the matter with MiniCozy on my next visit. I located the project manager for CozyWrite, and he acknowledged the existence of the error.
"It's a rare bug," he said.
"I wouldn't say so," I countered. "I found over twenty-five instances in her book."
"But it would only happen in a book-sized project. Out of over 100,000 customers, we probably didn't have 10 who undertook a project of that size as a single file."
"But my niece noticed. It was her first book, and she was devastated."
"Naturally I'm sorry for her, but it wouldn't have made any sense for us to try to fix the bug for 10 customers."
"Why not? You advertise that CozyWrite handles book-sized projects."
"We tried to do that, but the features didn't work. Eventually, we'll probably fix them, but for now, chances are we would introduce a worse bug–one that would affect hundreds or thousands of customers. I believe we did the right thing."
As I listened to this project manager, I found myself caught in an emotional trap. As software consultant to MiniCozy, I had to agree, but as uncle to an author, I was violently opposed to his line of reasoning. If someone at that moment had asked me, "Is CozyWrite a quality product?" I would have been tongue-tied.
How would you have answered?

The Relativity of Quality
The reason for my dilemma lies in the relativity of quality. As the MiniCozy story crisply illustrates, what is adequate quality to one person may be inadequate quality to another.

Finding the relativity
If you examine various definitions of quality, you will always find this relativity. You may have to examine with care, though, for the relativity is often hidden, or at best, implicit.
Take for example Crosby's definition:
"Quality is meeting requirements."
Unless your requirements come directly from heaven (as some developers seem to think), a more precise statement would be:
"Quality is meeting some person's requirements."
For each different person, the same product will generally have different "quality," as in the case of my niece's word processor. My MiniCozy dilemma is resolved once I recognize that
a. To Terra, the people involved were her readers.
b. To MiniCozy's project manager, the people involved were (the majority of) his customers.

Who was that masked man?
In short, quality does not exist in a non-human vacuum, but every statement about quality is a statement about some person(s).
That statement may be explicit or implicit. Most often, the "who" is implicit, and statements about quality sound like something Moses brought down from Mount Sinai on a stone tablet. That's why so many discussions of software quality are unproductive: It's my stone tablet versus your Golden Calf.
When we encompass the relativity of quality, we have a tool to make those discussions more fruitful. Each time somebody asserts a definition of software quality, we simply ask,
"Who is the person behind that statement about quality."
Using this heuristic, let's consider a few familiar but often conflicting ideas about what constitutes software quality:

a. "Zero defects is high quality."
1. to a user such as a surgeon whose work would be disturbed by those defects
2. to a manager who would be criticized for those defects

b. "Lots of features is high quality."
1. to users whose work can use those features–if they know about them
2. to marketers who believe that features sell products

c. "Elegant coding is high quality."
1. to developers who place a high value on the opinions of their peers
2. to professors of computer science who enjoy elegance

d. "High performance is high quality."
1. to users whose work taxes the capacity of their machines
2. to salespeople who have to submit their products to benchmarks

e. "Low development cost is high quality."
1. to customers who wish to buy thousands of copies of the software
2. to project managers who are on tight budgets

f. "Rapid development is high quality."
1. to users whose work is waiting for the software
2. to marketers who want to colonize a market before the competitors can get in

g. "User-friendliness is high quality."
1. to users who spend 8 hours a day sitting in front of a screen using the software
2. to users who can't remember interface details from one use to the next

The Political Dilemma
Recognizing the relativity of quality often resolves the semantic dilemma. This is a monumental contribution, but it still does not resolve the political dilemma:
More quality for one person may mean less quality for another.
For instance, if our goal were "total quality," we'd have to do a summation over all relevant people. Thus, this "total quality" effort would have to start with a comprehensive requirements process that identifies and involves all relevant people. Then, for each design, for each software engineering approach, we would have to assign a quality measure for each person. Summing these measures would then yield the total quality for each different approach.
In practice, of course, no software development project ever uses such an elaborate process. Instead, most people are eliminated by a prior process that decides:
Whose opinion of quality is to count when making decisions?
For instance, the project manager at MiniCozy decided, without hearing arguments from Terra, that her opinion carried minuscule weight in his "software engineering" decision. From this case, we see that software engineering is not a democratic business. Nor, unfortunately, is it a rational business, for these decisions about "who counts" are generally made on an emotional basis.

Quality Is Value To Some Person
The political/emotional dimension of quality is made evident by a somewhat different definition of quality. The idea of "requirements" is a bit too innocent to be useful in this early stage, because it says nothing about whose requirements count the most. A more workable definition would be this:
"Quality is value to some person."
By "value," I mean, "What are people willing to pay (do) to have their requirements met." Suppose, for instance, that Terra were not my niece, but the niece of the president of the MiniCozy Software Company. Knowing MiniCozy's president's reputation for impulsive emotional action, the project manager might have defined "quality" of the word processor differently. In that case, Terra's opinion would have been given high weight in the decision about which faults to repair.

The Impact on Agile Practices
In short, the definition of "quality" is always political and emotional, because it always involves a series of decisions about whose opinions count, and how much they count relative to one another. Of course, much of the time these political/emotional decisions–like all important political/emotional decisions–are hidden from public view. Most of us software people like to appear rational. That's why very few people appreciate the impact of this definition of quality on the Agile approaches.
What makes our task even more difficult is that most of the time these decisions are hidden even from the conscious minds of the persons who make them. That's why one of the most important actions of an Agile team is bringing such decisions into consciousness, if not always into public awareness. And that's why development teams working with an open process (like Agile) are more likely to arrive at a more sensible definition of quality than one developer working alone. To me, I don't consider Agile any team with even one secret component.
Customer support is another emphasis in Agile processes, and this definition of quality guides the selection of the "customers." To put it succinctly, the "customer" must actively represent all of the significant definitions of "quality." Any missing component of quality may very likely lead to a product that's deficient in that aspect of quality. As a consultant to supposedly Agile teams, I always examine whether or not they have active participation of a suitable representation of diverse views of their product's quality. If they tell me, "We can be more agile if we don't have to bother satisfying so many people, then they may indeed by agile, but they're definitely not Agile.


7 comments:

Unknown said...

I consider the lines:

c. "Elegant coding is high quality."
1. to developers who place a high value on the opinions of their peers

a little disparaging

Could we say instead
c. "Elegant coding is high quality."
1. to future developers who have to return to the code to maintain or develop it further.

Unknown said...

I read your book on quality software management in the mid nineties and loved it. However, I have always had trouble, not with your definition of what you call quality, but your choice of the word "quality." At at least one place that I worked, rather than help understand requirements it led to more or less bike-shedding arguments about the meaning of the word.

I worked for a wall street program trading (index arbitrage) company. There was a window of, say, two weeks in which they needed a way to price a certain kind of product. The IT department was broken down into research (mathematicians) and systems (programmers). The software to do the pricing was written by someone in the research department. It was a command line program written in C to run under Unix. About half the time it was run it printed "segmentation fault, core dumped" and exited. The author of the program said, "Don't worry about that, just run it again." (I never personally analyzed the code, so I have no idea why rerunning it fixed the problem).

There were a number of high priority projects going on, and no one in systems was available to spend time on this. The guy who wrote the program was not up to the task of finding the bug. The program seemed to work reliably when it worked at all. My main concern about releasing it into production was that it lowered the standards of what people would accept.

At any rate, here's the problem with talking about this program in terms of "quality." The subroutine that did the real work was clearly of pretty high quality in the sense that it took in the necessary information and spat out what the traders needed to know. The packaging of the program was abominable. No one at the company, even the traders who needed it, would have referred to that piece of software as high quality. However, no one would have asked the programmer to spend time fixing it. It was needed "now" and for the coming two weeks. After that it would have no value at all.

While no one at the company would have said that program was high quality, everyone knew that it made reasonable tradeoffs for its intended purpose. So, I think the problem here is that recognizing "value to some person" is important, possibly more important than "quality," but "quality" is possibly not the best word to use to describe it.

Peter Varhol said...

At the very least, MiniCozy can be faulted for false or deceptive marketing. That may not be the project manager's problem, but it is the company's.

Great post, it illustrated a point I had tried to make with much less success recently.

Yaki Koren said...

The golden calf was there because the "who" was unreachable. When the customers are far away we start making assumptions which in time become facts in our minds.

Anonymous said...

A point well made. Reminded me of your "quality versus apple pie" story.

Sadly, many teams just jumping on the agile bandwagon focus on the wrong things. They pride themselves in number of user stories delivered, velocity, or other numbers, at the expense of quality, because it is not straightforward or because it doesn't fit their user story template.

I'm delighted you're writing on the subject; hopefully these posts will help inspire them to give it some more attention.

David said...

It seems there is a shift in agile community so now all teams must release a high quality / low defects product. Why we don't let our product owner and customer decide what level of quality they feel most comfortable with?

Kouros Mohit said...

Product "quality" can only be viewed from the consumer's (enduser, customer) point of view. If the product fails to perform any of the required functions per specification then it may not be qualified as high quality. It is as simple as that.