Sunday, November 26, 2017

How Do I Decide Between appX and appY?

Hardly a day goes by without some developer or tester asking me about some tools or applications. These could be any tools or apps, so let's call them X and Y.

Usually, the question is simple, but asked with heart-stopping urgency:

"Is X better than Y?"

Rather than provide an answer, I tell them they would be better off not asking such "better than?" questions.

Software apps and tools are complex systems. Consequently any X-Y pair will differ on a number of dimensions. X will be better on some; Y will be better on others. Or both will be useless or poor for your needs.

If you're choosing a tool or an app, start with assessing your needs. Then, instead of asking which is better, ask

"Which fits my needs better, X or Y?"

If neither one fits you needs, then look for a third alternative, or a fourth.

In the rare case when both X and Y fit your needs, you might meaningfully ask, "Which is better—for me, at this moment?"

If X and Y still seem equal, then flip a coin. Heads, take X. Tails, take Y.

Then, while the coin is in the air, your mind will usually make the decision, not willing to allow the coin drop to make the decision for you.

But, if your mind doesn't decide, then let the coin drop decide. At that point, it shouldn't matter.

But if you reach this point, wait a moment before you choose X or Y. During that moment, consider the following two questions:

Can I take both X and Y?

What about Z? Is there some third alternative I haven't considered?

Indeed, instead of asking "which is better" questions, ask, "What is the problem I'm trying to solve?"

Are Your Lights On?: How to know what the problem really is?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          

Sunday, November 19, 2017

Terchnical Reviews and Organizational Renewal

We know that success can breed failured and doesn't automatically renew itself. I would like to offer some ideas on how this self-defeating tendency can be resisted.

One way toward renewal is through new perspectives gained from outside technical audits, but audits suffer from several serious disabilities. For one thing, audits are done intermittently. In between one audit and the next, the programmers don't stop programming, the analysts don't.stop analyzing, and the operators don't stop operating. Sometimes, though, the managers stop managing, And there's the rub.

A comparable situation exists when a firm has a system of personnel reviews mandated, say, every six months for each employee. Under such a system, managers tend to postpone difficult interactions with an employee until the next appraisal is forced upon them. A huge dossier may be accumulated, but then ignored in favor of the last, most conspicuous blunder or triumph. Good managers realize that the scheduled personnel review is, at best, a backup device—to catch those situations in which day-to-day management is breaking down.

In the same way, the outside technical audit merely backs up the day-to-day technical renewal processes within the shop. It may prevent utter disasters, but it's much better if we can establish technical renewal on a more continuous and continuing basis. One way to do this is through a technical team, such as an Agile team. For now, though, I want to introduce, or reintroduce, the concept of formal and informal technical reviews as tools for renewing the technical organization.

The Agile Manifesto requires "technical excellence" and "simplicity," and states: 

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. 

To achieve these and other goals, Agile teams conduct "walkthroughs" and "inspections," but these are only two very specific examples of technical review procedures used by Agile teams. In my reading and consulting, I've uncovered dozens of major variants of these two types of review, plus literally hundreds of minor variants that shops have introduced to make the review more suitable to their environments, whether or not they claim to be "Agile."

A more general definition of a technical review could be 

1. a review of technical material on the basis of content (this is what makes it a "technical" review, rather than, say, a financial or personnel review)

2. which is done openly by at least two persons (this is what distinguishes it from "desk checking")

3. who take lull responsibility for the quality of the review (not, please note, for the quality of the product)

Then we distinguish between the informal and formal review. A formal review produces a written report to management. That report is the formal part.

Informal reviews (that is, reviews which need not terminate with a written report to management) are also excellent devices for an organization's self-renewal. Informal reviews take all forms of technical reviews, and are practiced everywhere almost all the time. Indeed, they are an essential part  of the real world of programming work.

For instance, a team member passes a diagram to a teammate for an opinion on how to represent a particular design. Someone asks help from someone else in finding an error. A set of test cases is passed around to see if anyone can think of something that's been overlooked. One person reads another's user document to see if it's understandable.

Without a constant flow of such informal reviewing of one another's work, programming any meaningful project would be impossible. Formal reviewing, on the other hand, is not essential to programming. Many small projects are accomplished without formal reviewing, which is why some programmers don't appreciate the need for formal reviewing. 

As projects grow 'larger and more complex—as they are inclined to do in successful shops—the work of many people must be coordinated over a long period of
time. Such coordination requires management—though not necessarily managers—and such management requires reliable information. Formal reviews, essentially, are designed to provide reliable information about technical matters—particularly to non-technical people.

Clearly, a successful system of formal technical reviews—that is, a system that provides management with reliable information on technical matters—is essential to management participation in the organizational-renewal process. For the large shop, formal reviews provide information that the small shop manager gets "through the seat of the pants." Many, many failures of previously successful programming organizations an be traced directly to the breakdown of the earlier informal mechanisms for communicating reliable information about technical matters.

There may, of course, be other ways of getting this kind of information, and many smaller shops do an excellent job without any system of formal reviews. Even those shops, however, may benefit from an explicit system of reviews to supplement their implicit, or informal, existing system. 

Principally, the formal technical review provides reliable self-monitoring. An informal system may work just as well as a formal one, and if so, there are many reasons to prefer to keep the reviewing informal. In any case, there will always be an informal system at least supplementing the formal one, but we should really view the formal system as supplementing the informal. Its formality guards against creeping deterioration of the organization.

Regular reviews of specifications, designs, code, test plans, documentation, training materials, and other technical matter have many beneficial "side effects" on the health and success of an installation. Reviews have a very marked effect on maintenance—that quicksand in which a thousand successful shops have met an untimely end. A typical program that has been thoroughly reviewed during its development will

1. require less maintenance work per change

2. require fewer changes caused by poor specification, design, coding, or testing.

Instituting a program of technical reviews will not, of course, have any immediate effect on the existing burden of maintenance carried like an albatross from a sinful programming past. Indeed, when maintenance programmers participate in reviews of newer code, they may be further discouraged by the poor quality of the code with which they are burdened. But, the situation can improve rather quickly, for a variety of reasons:

1. Some new ideas can be applied even to patches to old programs, though naturally limited by poor existing structure and style.

2. Through mutual participation in reviews, the entire shop quickly obtains a realistic and sympathetic picture of the true dimensions of the maintenance situation.

3. The worst maintenance problems will, through frequent reviews, become exposed to fresh air and sunlight.

Quite frequently, installation of a review system is quickly followed by replacement of the most cantankerous old beasts left over from the Stone Age of Programming. The effect of even one such replacement on maintenance morale is a wonder to behold,

Other activities besides maintenance are similarly affected. In the long run, certainly, reviews have a remarkable effect on staff competence, perhaps the most important element in continuing success. We also see increased morale and professional attitude, reduced turnover, more reliable estimating and scheduling, and better appreciation of the management role in project success. (Usually, the technical staff has had no difficulty in appreciating the management role in project failure.)

At the same time, reviews provide management with a better appreciation for staff competence, both individually and collectively. The unsung hero who quietly saved a dozen shaky projects is now sung far and wide. The "genius programmer" who was always the darling of the executives has been exposed for the empty and obsolete shell the technical staff always knew, but couldn't expose to management.

No other factor is more depressing to a technical organization than misappraisal of technical competence by non-technical management. The openness of technical reviews marks an end to that misappraisal era. No longer will we all feel cheated by charlatans and incompetents.

As a consultant, I visited dozens of installations every year. The best way to summarize the effects of properly instituted reviews is to report that after five minutes in an installation, I can tell—without asking directly—to what extent there is an effective review-practice, formal or informal.

How do I tell? Metaphorically, I tell in the same way a government health inspector tells about a food processing plant—by the way it smells. It's hard to describe, but when you smell the difference, you-know it!

* * * * * *

Looking back over this essay, I sense its inadequacy to the task at hand. Those programmers and analysts who have experienced a shop with a properly functioning system of reviews will know all this without my giving any details. They've experienced it, and if they are professionals, they'll never agree to work in any other kind of environment.

But those who have never experienced such an environment of a self-renewing organization will not understand, or will misunderstand, what I've written. Some of them will have experienced a misguided attempt to institute reviews. Perhaps the attempt was in the form of a punitive management mandate. Perhaps it was merely a case of another programmer who read one article and blundered ahead with 99% enthusiasm and 1% information—and zero human feeling. To these people, the experience of "reviews" will have left a bitter taste, or a painful memory. They will not recognize their experience in what I've written.

In many ways, technical reviewing is like bicycling. Up until the time you first discover your balance in one miraculous instant, the whole idea seems impossible, contrary to nature, and a good way to break a leg. Indeed, if you'd never seen a cyclist before, and had the process described to you, you'd most certainly declare the whole procedure impossible. And, without instruction and encouragement from experienced cyclists, along with reliable "equipment," the most likely outcome would certainly be skinned knees, torn pants, and a few lumps on the head. And so it has been with technical reviews—until now—-so don't go off and try to learn the whole process on your own.

If you want to get started renewing the success of your own shop through a system of technical reviews, find an experienced shop, or a person from an experienced shop, to guide you. Listen to them, Read the literature. Then try a few simple reviews on an experimental basis, without much fanfare.

Adapt the "rules" to your own environment. Be forgiving and humane. Your rewards  will not be long in coming.


Wednesday, November 15, 2017

What's it like to rewrite a program from scratch?

This is an interesting question because so many programmers are so afraid of this task that they would never even ask it. Is this reluctance agile, or Agile?

But why would you want to do rewrite a program from scratch? The most important reason is to increase maintainability. In the initial writing, the focus is generally on merely getting the job done, without thinking of the program's future. Over their lifetime, many programs cost far more to maintain than to write originally, especially when the original program has become a thing of rags and patches.

A second, and often secondary reason to rewrite a program is efficiency. Newly constructed programs and highly patched old programs sometimes turn out to be slower than desired, but such inefficiency cannot erased by any amount of tweaking. Instead, increased efficiency requires a new approach, an approach that needs to be implemented from scratch.

But isn't rewriting expensive? Not usually. In fact, it’s generally far, far easier to rewrite a program from scratch than to write some brand-new program.

Why would a fresh start-over be cheaper than the original? Because in writing the original program, the original programmers answered so many difficult questions about what was really wanted. Requirements haven't changed, and most of the thought put into testing can be reused.

Those questions—requirements and test—usually make up more than half the total work put into a program. They have already been answered—but only if you resist the temptation to change those answers. If you don’t resist, then rewriting the program can be arbitrarily difficult.

I wish more programmers had to courage to rewrite some clumsy programs from scratch, rather than patch and patch and patch. And I wish their managers would encourage sensible rewriting, rather than force programmers to waste their skills, time, and energy keeping ancient programs on life support.

Tuesday, November 07, 2017

When do I know I'm not a beginning programmer any more?

I was asked, "When do I know I'm not a beginning programmer any more?"

I wouldn’t answer this question, because it’s the wrong question.

You should not ever want to know you’re not a beginner, because a true professional is always a beginner. The world in general, and the world of programming in particular, is so complex, so huge, that one lifetime is not long enough to stop being a beginner.

Your beginner’s mind is one of your most valuable tools. It requires you to look at each situation afresh, and to innovate. (Fundamentally, that's what the Agile movement is all about.) If you know children, watch how they use their beginner’s minds to conquer their world.

I’m very suspicious of people in the programming field who think they are no longer beginners. Myself, I’ve been programming for about 70 years, and I still consider myself a beginner.

Thursday, November 02, 2017

How do I become a programmer who gets stuff done?

The young man wanted to know, "How do I become a programmer who gets stuff done?" He received a number of good answers, like how to organize his work and schedule his working hours. Yet I had a different view of things, so I gave a different sort of answer, as follows.

Some good advice here, yet even if you do all these suggested things, you may be having a different problem. When you speak of getting stuff done, you may be speaking of “finishing” things.

Defining what “done” means has been a classic programmer problem for more than 50 years. Part of the problem is that programmers with different personalities have different ideas about what “done” means. For instance, look at the situation from the point of view of MBTI personality temperaments:

NTs tend to think a project is done when they have a clear description of the problem and a general approach to solving it. Someone else can work out the details.

SJs tend to think a project is done when it’s “code complete”—though research at Microsoft and other places seems to indicate that only about two-thirds of the eventual code has been written by that supposed benchmark. Perhaps this work isn't thought of real programming, but only "clean up."

NFs, such as me, tend to think that a project is done when everyone involved is satisfied that it’s done. Of course, because these "others" are of various personality types, our NF estimate of "done" isn't very reliable.

SPs tend to think a project is done when they are bored with doing it. They share this feeling with lots of other temperaments, too. Programmers in general don't tolerate boredom very well.

Not taking that last step to "clean up" is a curse of our profession, leaving many programs in a less-than wholesome state. Unclean, unfinished programs are the source of many maintenance problems, and of many errors shipped to customers.

Of course, this classification is only a rough model of non-finishers. Many good programmers of all temperaments are excellent finishers, having taught themselves to be aware of their tendencies and counter them with a variety of tactics. Primary among those tactics if the technical review by peers (which is an integral part of the Agile approach but  by no means is not confined to Agile).

So, you offer your “finished” project to your peers for review, and if they agree that it’s finished, you’ve truly gotten something “done.”

If they don’t agree that your work is done, they will give you a list of issues that you need to address before you’re “done.” You then go back to work and address these issues, then resubmit the newest version to another technical review.

Finally, you iterate in this reviewing process until your work passes the review. Then you know you’ve gotten something done.

For more detail on the review process, see,