How do I fix a really difficult bug in programming?

Here was the question:

"How do I fix a really difficult bug in programming?"

Here was my first answer:

There is no such thing as a “difficult bug.”

I suspect my answer requires further explanation. First of all, I doubt that you have experienced actual bugs in your computer, the kind with 8 legs that bite and swarm. I have, a couple of times, but they are rare, and usually not difficult to eradicate.

Perhaps you are talking about errors, but using inaccurate language. In that case, I will assert “there is no such thing as a difficult error.” The same error might be handled easily by a different person. I have seen that circumstance often. For instance, I once spent a month trying to pinpoint a coding error. When I finally asked the help of a colleague, she found it in less than two minutes.

No, there are no difficult errors, but there are people who have difficulty with an error. We have all been there, and we tend to want to blame the error rather than ourselves.

So, the first thing you need to do to handle a “difficult bug” is to ask yourself,

“What is it about me that is making this error so difficult to handle?”

Perhaps you are having difficulty because you are impatient, or think failure to handle the error will make you look bad to your boss or colleagues.

Perhaps pressure to handle the error is throwing you off your center, distorting your thinking.

Perhaps you do not know enough about the system with the error, or the language in which the program is written.

Perhaps your mind is on other things in your life, things distracting you because they are more important to you than this darn “bug.”

Maybe you should discuss this error with a colleague or two, What is it about you that is keeping you from doing that?

Anyway, good luck in your quest for resolution.

Take a look at Errors: Bugs, Boo-Boos, and Blunders

New: #System #Design #Heuristics

You'd think that after publishing books for half a century, I'd know how to write a book. If that's what you think, you'd be wrong.

Sure, I've even written a book on writing books (Weinberg on Writing, the Fieldstone Method), and I've applied those methods to dozens of successful books. But way back around 1960, I started collecting notes on the process of design, thinking I would shortly gather them into a book. Back then, I didn't call these bits and pieces "fieldstones," but that's what they turned out to be: the pieces that, when assembled properly, would ultimately become my design book.

Ultimately? Assembled properly? Aye, there's the rub!

Building walls from randomly found fieldstones requires patience. So does writing books by the Fieldstone Method. My Introduction to General Systems Thinking took fourteen years to write. But a writer only lives one lifetime, so there's a limit to patience. I'm growing old, and I'm beginning to think that fifty years is as close to "ultimately" as I'm going to get.

So, I've begun to tackle the task of properly assembling my collection of design fieldstones. Unfortunately, it's a much larger collection that I'd ever tackled before. My Mac tells me I have more than 36,000,000 digitized bytes of notes. My filing cabinets told me I had more than twenty-five pounds of paper notes, but I've managed to digitize some of them and discard others, so there's only a bit more than ten pounds left to consider.

For the past couple of years, I've periodically perused these fieldstones and tried to assemble them "properly." I just can't seem to do it. I'm stuck.

Some writers would say I am suffering from "writer's block," but I believe "writer's block" is a myth. I've published three other books in these frustrating years, so I can't be "blocked" as a writer, but just over this specific design book. You can hear me talk more about the Writer's Block myth on YouTube 

[https://www.youtube.com/watch?v=77xrdj9YH3M&t=7s]

but the short version is that "blocking" is simply a lack of ideas about how to write. I finally decided to take my own advice and conjure up some new ideas about how to write this design book.

Why I Was Stuck

To properly assemble a fieldstone pile, I always need an "organizing principle." For instance, my recent book, Do You Want to Be a (Better) Manager? is organized around the principle of better management. Or, for my book, Errors, the principle is actually the title.  So, I had been thinking the organizing principle for a book on design ought to be Design. 

Well, that seemed simple enough, but there was a problem. Everybody seemed to know what design is, but nobody seemed able to give a clear, consistent definition that covered all my notes. I finally came to the conclusion that's because "design" is not one thing, but many, many different things.

In the past, I ran a forum (SHAPE: Software as a Human Activity Practiced Effectively) whose members were among the most skilled software professionals in the world. We held a number of threads on the subject, "What is Design?" The result was several hundred pages of brilliant thoughts about design, all of which were correct in some context. But many of them were contradictory.

Some said design was a bottom-up process, but others asserted it was top-down. Still others talked about some kind of sideways process, and there were several of these. Some argued for an intuitive process, but others laid out an algorithmic, step-by-step process. There were many other variations: designs as imagined (intentional designs), designs as implemented, and designs as evolved in the world. All in all, there were simply too many organizing principles—certainly too many to compress into a title, let alone organize an entire book.

After two years of fumbling, I finally came up with an idea that couldn't have been implemented fifty years ago: the book will be composed of a variety of those consulting ideas that have been most helpful to my clients' designers. I will make no attempt (or very little) to organize them, but release them incrementally in an ever-growing ebook titled Design Heuristics.

How to Buy System Design Heuristics

My plan for offering the book is actually an old one, using a new technology. More than a century ago Charles Dickens released many of his immortal novels one chapter at a time in the weekly newspaper. Today, using the internet, I will release System Design Heuristics a single element at a time to subscribing readers.

To subscribe to the book, including all future additions, a reader will make a one-time payment. The price will be quite low when the collection is small, but will grow as the collection grows. That way, early subscribers will receive a bargain in compensation for the risk of an unknown future. Hopefully, however, even the small first collection will be worth the price. (If not, there will be a full money-back guarantee.)

Good designs tend to have unexpected benefits. When I first thought of this design, I didn't realize that it would allow readers to contribute ideas that I might incorporate in each new release. Now I aware of that potential benefit, and look forward to it.

Before I upload the first increment of System Design Heuristics, I'll wait a short while for feedback on this idea from my readers. If you'd like to tell me something about the plan, email me or write a comment on this blog.

Thanks for listening. Tell me what you think.

Which code is more readable?

We were asked, "Which code is more readable, one that uses longer variable names or short ones?" 

Maybe some historical perspective will help answer this question.

In the very early days of computing (I was there), we used short variable names because:

* Programs were fairly short and simple, so scope wasn’t much of a problem.

• Memories were small, so programmers didn’t want to waste memory with long names.

• Compilers and assemblers were slow, and long names made them slower.

• Many compilers and assemblers wouldn’t allow names longer than a few characters, because of speed and memory limitations.

• We didn’t think much, if at all, about who would maintain a program once it left the hands of the original programmer.

As programs grew larger, one result of short naming was difficult maintenance, so the movement toward longer names grew stronger. It wasn’t helped by COBOL, which asserted that executives should be able to read code. Lots of COBOL code was littered with super-long names, but that didn't help executives read it.

The COBOL argument proved to be nonsense. Still, the maintenance argument for longer, more descriptive names made sense.

Unfortunately, like many movements, the long-name movement went too far, at least for my taste. It wasn’t because long names were harder to write. After all, a typical program is written oncem but read for modification and testing many, many times. So, if long names really made reading easier and more reliable, it was good.

But the length of a name is not really the issue. I’ve seen many programs with long, long names that were so similar that they were easily confused, one with another. For instance, we once wasted many days trying to find an error when the name radar_data_station_#46395_azimuth_reading was mistaken for radar_data_station_#46895_azimuth_reading. Psychologists and writers know well that items in the middle of long lists are frequently glossed over.

So, like lots of other things in software development, long versus short names becomes a tradeoff, a design decision for a programmer for which there is no “right” answer. Programmers must design their name-sets with the same kind of engineering thought they put into all their design decisions.

And, as maintainers modify a program, they must maintain the name-set, so as to avoid building up design debt as the program ages.

So, sorry, there’s no easy answer to this question, nothing a programmer can apply  mindlessly. Just as it’s always been, programmers who think will do a better job than those who blindly follow simplistic rules.

The Psychology of Computer Programming: Silver Anniversary eBook Edition

Must There Always Be Inferior Code?

Some people claim that when you learn high software standards you will never again develop in inferior ways. Is this true?

I think you can arrive at a meaningful answer by using an analogy:

Some people claim that when you learn high medical standards, a doctor or nurse will never again treat a patient in inferior ways. Is that true?

Seen in this light, the answer is obvious. Most doctors and nurses will not treat patients in inferior ways—unless it's an emergency, like an explosion or a fire in which many people need saving in a hurry. If that happens, the doctor or nurse will return to those patients when the emergency has calmed down. Same in software.

But there do exist a few medical professionals who don’t live up to such high standards. They are, after all, human beings. Yet in spite of their inferior practices, some of their patients do get better. Why? Because humans have built-in healing mechanisms—but software does not.

Software with sick code doesn’t heal itself. Those programmers who develop in inferior ways will eventually produce troublesome code. But the key word is "eventually."

The inferior programmer may not be around any longer when the code's trouble makes itself known, so some inferior programmers can get away with hacky ways for an entire career.

It’s a good manager's job to recognize these inferior programmers and replace them and their code before the true costs of their inferior work become evident.

Some managers overuse the tactic of forcing programmers to code in a hurry, as if there's always an emergency. Just as in medicine, emergency treatment of code tends to produce inferior results. Managers who care only about the short-term will not do anything about their inferior programmers, but they, too, may move out before the consequences of their inferior management become apparent.

That’s why inferior programming practices persist. And, as long as programmers and managers are human, inferior practices will always persist. But they don't have to persist in your world. It's up to you. \

Code in haste, debug forever.

Survey: The Worst Error Message

I participated in a recent survey. The question was:

What's the Worse Error Message You've Ever Seen?

This was my contribution, which received thousands of up-votes:

A graduate student brought me an error message that said:

THIS IS AN IMPOSSIBLE ERROR. THIS ERROR CANNOT OCCUR.

IF THIS ERROR OCCURS, SEE YOUR IBM REPRESENTATIVE.

I told the student I wasn’t the right person to see, but he should see his IBM representative.

He said, “But I’m my IBM representative.”

—————————-

I'd like to collect some of these lousy error messages for my book, ERRORS.

If you've got a good (i.e. bad) one, please share it with us in a comment to this post.

Thanks for contributing.

What's the most complex thing about software development?

What's the most complex thing about software development?

Interesting question.

So far, on Quora.com, there have been four excellent answers to this question: discussing 

- the confusing role of people, 

-the requirements problems,

-the interactions with the physical world.

Each of these factors certainly makes software development more complex, and processes such as Agile are designed to cope with this complexity. But, the ultimate complexity factor is software testing.

Why testing? In the software development literature, testing is not usually treated as a glamorous part of development, but when we're testing, we're up against the Second Law of Thermodynamics, which warns us that perfection is ultimately unobtainable.

So, even if we absolutely knew all the requirements (which we can't, of course), kept all the human factors under control (also impossible), and knew exactly all the physical properties of the real world (once more, impossible), we would still never be able to perform the infinite number of tests to cover all possible situations.

In other words, the software could still surprise us at any time. That's what I call complexity.

Of course, we can still work hard to solve these other problems. On requirements, for instance, see our Exploring Requirements books.

But no matter how hard you try, you'll still be faced with the testing problem. To understand this problem and what you can do to reduce (but not eliminate) it, take a look at Perfect Software and Other Illusions about Testing.

Preventing a Software Quality Crisis

Abstract

Many software development organizations today are so overloaded with quality problems that they are no longer coping with their business of developing software. They display all the classic symptoms of overloaded organizations—lack of problem awareness, lack of self-awareness, plus characteristic behavior patterns and feelings. Management may not recognize the relationship between this overload and quality problems stemming from larger, more complex systems. If not, their actions tend to be counterproductive. In order to cure or prevent such a crisis, management needs to understand the system dynamics of quality.

Symptoms of Overload Due to Poor Quality

In our consulting work, we are often called upon to rescue software development operations that have somehow gotten out of control. The organization seems to have slipped into a constant state of crisis, but management cannot seem to pin the symptoms down to one central cause. Quite often, that central cause turns out to be overload due to lack of software quality, and lack of software quality due to overload.

Our first job as consultants is to study symptoms. We classify symptoms of overload into four general categories—lack of problem awareness, lack of self-awareness, plus characteristic patterns of behavior and feelings. Before we  describe the dynamics underlying these symptoms, lets look at some of them as they my be manifest in a typical, composite organization, which we shall call the XYZ corporation.

Lack of Problem Awareness

All organizations have problems, but the overloaded organization doesn't have time to define those problems, and thus has little chance of solving them:

1. Nobody knows what's really happening to them.

2. Many people are not even aware that there is a system-wide problem.

3. Some people realize that there is a problem, but think it is confined to their operation.

4. Some people realize that there is a problem, but think it is confined to somebody else's operation.

5. Quality means meeting specifications. An organization that is experiencing serious quality problems may ignore those problems by a strategy of changing specifications to fit what they actually happen to produce. They can then believe that they are "meeting specifications." They may minimize parts of the specification, saying that it's not really important that they be done just that way. Carried to an extreme, this attitude leads to ignoring certain parts of the specification altogether. Where they can't be ignored, they are often simply

forgotten.

6. Another way of dealing with the overload is to ignore quality problems that arise, rather than handling them on the spot, or at least recording them so others will handle them. This attitude is symptomatic of an  organization that needs a top-to-bottom retraining in quality.

Lack of Self-Awareness

Even when an organization is submerged in problems, it can recover if the people in the organization are able to step back and get a look at themselves, In the chronically overloaded organization, people no longer have the means to do this. They are ignorant of their condition, and they have crippled their means of removing their ignorance:

7. Worse than not knowing what is going on is thinking you know, when you don't, and acting on it. Many managers at XYZ believe they have a grip on what's going on, but are too overloaded to actually check. When the reality is investigated, these managers often turn out to be wrongs For instance, when quizzed about testing methods used by their employees, most managers seriously overestimate the quality of testing, when compared with the programmers' and testers' reports.

8. In XYZ) as in all overloaded organizations, communication within and across levels is unreliable and slow. Requests for one kind of information produce something else, or nothing at all. In attempting to speed up the work, people fail to take time to listen to one another, to write things down, or to follow through on requested actions.

9. Many individuals at XYZ are trying to reduce their overload by isolating themselves from their co-workers, either physically or emotionally. Some managers have encouraged their workers to take this approach, instructing them to solve problems by themselves, so as not to bother other people.

10. Perhaps the most dangerous overload reaction we observed was the tendency of people at XYZ to cut themselves off from any source of information that might make them aware of how bad the overload really is. The instant reaction to any new piece of information is to deny it, saying there are no facts to substantiate it. But no facts can be produced because the management has studiously avoided building or maintaining information systems that could contradict their claims that "we.just know what's going on." They don't know, they don't know they don't know, and they don't want to know they don't know. They're simply too busy.

Typical Behavior Patterns

In order to recognize overload, managers don't have to read people's minds. They can simply observe certain characteristic things they do:

11 . The first clear fact that demonstrates overload is the poor quality of the products being developed. Although it's possible to deny this poor quality when no measurements are made of the quality of work in progress, products already delivered have shown this  poor quality in an undeniable way.

12. All over the organization, people are trying to save time by short-circuiting those procedures that do exist. 'This tactic may occasionally work in a normal organization faced with a short-term crisis, but in XYZ, it has been going on for so long it has become part of standard operating procedure.

13. Most people are juggling many things at one time, and thus adding coordination time to their overload. In the absence of clear directives on what must be done first, people are free to make their own choices. Since they are generally unaware of the overall goals of the organization, they tend to suboptimize, choosing whatever looks good to them at the moment.

14. In order to get some feeling of accomplishment, when people have a choice of tasks to do, they tend to choose the easiest task first, so as to "do something." This decision process gives a short-term feeling of relief, but in the long term results in an accumulation of harder and harder problems.

15. Another way an individual can relieve overload is by passing problems to other people. As a result, problems don't get solved, they merely circulate. Some have been circulating for many months.

16. Perhaps the easiest way to recognize an overloaded organization is by noticing how frequently you hear People say, in effect, that they recognize that the way they're working is wrong, but they "have no time to do it right." This seems almost to be the motto of the XYZ organization.

Typical Feelings

If you wait for measurable results of overload, it may be too late. But its possible to recognize an overloaded organization through various expressions of peoples' feelings:

17. An easy way to recognize an overloaded organization is by the general atmosphere in the workplace. In many areas at XYZ there was no enthusiasm, no commitment, and no intensity. People were going through the motions of working, with no hope of really accomplishing their tasks.

18. Another internal symptom of overload is the number of times people expressed the wish that somehow the problem would just go away. Maybe the big customer will cancel the contract. Maybe the management will Just slip the schedules by a year. Maybe the sales force will stop taking more orders. Maybe the company will fail and be purchased by a larger company.

19. One common way of wishing the problem would go away is to choose a scapegoat, who is the personified source of all the difficulties, and then wish that this person would get transferred, get fired, get sick, or quit. At XYZ, there are at least ten different scapegoats—some of whom have long gone, although the problems still remain.

20. Perhaps the ultimate emotional reaction to overload is the intense desire to run away. When there are easy alternatives for employees, overload is followed by people leaving the organization, which only increases the overload. The most perceptive ones usually leave first. When there are few attractive opportunities outside, as at XYZ, then people "run away" on the job. They fantasize about other jobs, other places, other activities, though they don't act on their fantasies. Their bodies remain, but their hearts do not.

The Software Dynamics of Overload

There are a number of reasons for the overload situation at organizations like XYZ, but underlying everything is the quality problem, which in turn arises from the changing size and complexity of the work. This means that simple-minded solutions like adding large numbers of people will merely make the problems worse. In order for management to create a manageable organization, they will have to understand the dynamics of quality. In particular, they will have to understand how quality deteriorates, and how it has deteriorated in their organization over the years. The XYZ company makes an excellent case study.

The quality deterioration at XYZ has been a gradual effect that has crept up unnoticed as the size and complexity of systems has increased. The major management mistake has been lack of awareness of software dynamics, and the need for measurement if such creeping deterioration is to be prevented.

The quality deterioration experienced at XYZ is quite a common phenomenon in the software industry today, because management seems to make the same mistakes everywhere—they assume that the processes that would produce quality small systems will also produce quality large systems. But the difficulty of producing quality systems is exponentially related to system size and complexity, so old solutions quickly become inadequate. These dynamics have been studied by a number of software researchers, but it is not necessary to go fully into them here. A few examples will suffice to illustrate specifically what has been happening at XYZ and the kind of actions that are needed to reverse the situation.

NOTE: The remaining two-thirds of this article describes these dynamics and corrective actions, and can be found as a new chapter in the book, 

Do You Want To Be A (Better) Manager?

The book also details a number of the most common and distressing management problems, along with dozens of positive responses available to competent managers.