How can I become a software developer who only designs?

A young programmer asked me, "How can I become a software developer who only designs the whole software architecture and gives instructions to other developers rather than actually coding by myself?"

I told him he could do that right away, as long as he didn’t care if the other developers listened to his instructions and followed them. And if he didn't care of he was paid.

In general, software developers will not follow the lead of someone for whose designs they have no respect. And why would they respect your designs unless you had previously proved yourself by what you’ve built?

So, build things, and build more things, until you demonstrate that others have some reason to follow your lead.

And, at the same time, work on your people skills, because even if you’re the greatest designer in the world, if you’re a self-centered jerk, nobody will follow you or your designs.

For more on designing, see

System Design Heuristics

Unnecessary Code

We were asked, "How can I tell if my code does extra unnecessary work?"

To answer this question well, I’d need to know what you mean by “unnecessary.” Not knowing your meaning, I’ll just mention one kind of code I would consider unnecessary: code that makes your program run slower than necessary but can be replaced with faster code.

To rid your program of such unnecessary code, start by timing the program’s operations. If it’s fast enough, then you’re done. You have no unnecessary code of this type.

If it’s not fast enough, then you’ll want to run a profiler that shows where the time is being spent. Then you search those areas (there can be only one that consumes more than half the time) and work it over, looking first at the design.

There’s one situation I’ve encountered where this approach can bring you trouble. Code that’s fast enough with some sets of data may be unreasonably slow with other sets. The most frequent case of this kind is when the algorithm’s time grows non-linearly with the size of the data. To prevent this kind of unnecessary code, you must do your performance testing with (possibly artificially) large data sets.

Paradoxically, though, some algorithms are faster with large data sets than small ones.

Here’s a striking example: My wife, Dani, wanted to generate tests in her large Anthropology class. She wanted to give all students the same test, but she wanted the questions for each student to be given in a random order, to prevent cheating by peeking. She gave 20 questions to a programmer who said he already had a program that would do that job. The program, however, seemed to fall into an unending loop. Closer examination eventually showed that it wasn't an infinite loop, but would have finally ended about the same time the Sun ran out of hydrogen to burn.

Here’s what happened: The program was originally built to select random test questions from a large (500+ questions) data base. The algorithm would construct a test candidate by choosing, say, twenty questions at random, then checking the twenty to see if there were any duplicates among those chosen. If there were duplicates, the program would discard that test candidate and construct another.

With a 500 question data base, there was very little chance that twenty questions chosen at random would contain a duplicate. It could happen, but throwing out a few test candidates didn’t materially affect performance. But, when the data base had only twenty questions, and all Dani wanted was to randomize the order of the questions, the situation was quite different.

Choosing twenty from twenty at random (with replacement) was VERY likely to produce duplicates, so virtually every candidate was discarded, but the program just ground away, trying to find that rare set of twenty without duplicates.

As an exercise, you might want to figure out the probability of a non-duplicate set of twenty. Indeed, that’s an outstanding way to eliminate unnecessary code: by analyzing your algorithm before coding it.

Over the years, I’ve seen many other things you might consider unnecessary, but which do no harm except to the reputation of the programmer. For example:

* Setting a value that’s already set.

* Sorting a table that’s already sorted.

* Testing a variable that can have only one value.

These redundancies are found by reading the program, and may be important for another reason besides performance. Such idiotic pieces of code may be indications that the code was written carelessly, or perhaps modified by someone without full understanding. In such cases, there’s quite likely to be an error nearby, so don’t just ignore them.

Read: The Psychology of Computer Programming: Silver Anniversary eBook Edition 

Do programmers really know how to program?

I was asked, "Do programmers really know how to program?"

I believe this question is unproductive and  vague. What does it mean by “program”?

The person who asked this question seemed to think programmers were not really programming when all they did was copy some existing program, using it whole or perhaps pasting it in as part of a shell.

To me, programming a computer means instructing it to do something you want done, and to continue doing it as desired.

If that’s what we’re asking about, then yes, of course, some of us out here know how to program. (Some do not, of course.)

It is irrelevant how we do that. Whether we use genetic algorithms, cut-and-paste, or divine inspiration? Do we use Scrum or Agile or Waterfall? How about the programming language? C++, or Java, or Lisp, or Python, or APL? Well, none of those choices matters.

Then what does matter? How about, "Can we satisfy someone’s desires?" In other words, can we provide something that someone wants enough to pay what it costs, in time or money? That’s what counts, and we certainly know how do that—sometimes.

Sure, we fail at times, and probably too often. But no profession succeeds in satisfying its customers all the time. Did your teachers always succeed in teaching you something you wanted to know? Do surgeons know how to do surgery?

So what about using existing programs? To my mind, the first and foremost job of a programmer is knowing when not to write a program at all—either because the needed program already exists or because no program was needed in the first place.

In other words, not writing a program when no program is needed is the highest form of programming, and one of the marks of a true expert.

The Psychology of Computer Programming: Silver Anniversary eBook Edition 

or Kindle for the book in paper or ebook format

What's it like to rewrite a program from scratch?

This is an interesting question because so many programmers are so afraid of this task that they would never even ask it. Is this reluctance agile, or Agile?

But why would you want to do rewrite a program from scratch? The most important reason is to increase maintainability. In the initial writing, the focus is generally on merely getting the job done, without thinking of the program's future. Over their lifetime, many programs cost far more to maintain than to write originally, especially when the original program has become a thing of rags and patches.

A second, and often secondary reason to rewrite a program is efficiency. Newly constructed programs and highly patched old programs sometimes turn out to be slower than desired, but such inefficiency cannot erased by any amount of tweaking. Instead, increased efficiency requires a new approach, an approach that needs to be implemented from scratch.

But isn't rewriting expensive? Not usually. In fact, it’s generally far, far easier to rewrite a program from scratch than to write some brand-new program.

Why would a fresh start-over be cheaper than the original? Because in writing the original program, the original programmers answered so many difficult questions about what was really wanted. Requirements haven't changed, and most of the thought put into testing can be reused.

Those questions—requirements and test—usually make up more than half the total work put into a program. They have already been answered—but only if you resist the temptation to change those answers. If you don’t resist, then rewriting the program can be arbitrarily difficult.

I wish more programmers had to courage to rewrite some clumsy programs from scratch, rather than patch and patch and patch. And I wish their managers would encourage sensible rewriting, rather than force programmers to waste their skills, time, and energy keeping ancient programs on life support.

The Psychology of Computer Programming: Silver Anniversary eBook Edition 

When do I know I'm not a beginning programmer any more?

I was asked, "When do I know I'm not a beginning programmer any more?"

I wouldn’t answer this question, because it’s the wrong question.

You should not ever want to know you’re not a beginner, because a true professional is always a beginner. The world in general, and the world of programming in particular, is so complex, so huge, that one lifetime is not long enough to stop being a beginner.

Your beginner’s mind is one of your most valuable tools. It requires you to look at each situation afresh, and to innovate. (Fundamentally, that's what the Agile movement is all about.) If you know children, watch how they use their beginner’s minds to conquer their world.

I’m very suspicious of people in the programming field who think they are no longer beginners. Myself, I’ve been programming for about 70 years, and I still consider myself a beginner.

The Psychology of Computer Programming: Silver Anniversary eBook Edition

How do I become a programmer who gets stuff done?

The young man wanted to know, "How do I become a programmer who gets stuff done?" He received a number of good answers, like how to organize his work and schedule his working hours. Yet I had a different view of things, so I gave a different sort of answer, as follows.

Some good advice here, yet even if you do all these suggested things, you may be having a different problem. When you speak of getting stuff done, you may be speaking of “finishing” things.

Defining what “done” means has been a classic programmer problem for more than 50 years. Part of the problem is that programmers with different personalities have different ideas about what “done” means. For instance, look at the situation from the point of view of MBTI personality temperaments:

NTs tend to think a project is done when they have a clear description of the problem and a general approach to solving it. Someone else can work out the details.

SJs tend to think a project is done when it’s “code complete”—though research at Microsoft and other places seems to indicate that only about two-thirds of the eventual code has been written by that supposed benchmark. Perhaps this work isn't thought of real programming, but only "clean up."

NFs, such as me, tend to think that a project is done when everyone involved is satisfied that it’s done. Of course, because these "others" are of various personality types, our NF estimate of "done" isn't very reliable.

SPs tend to think a project is done when they are bored with doing it. They share this feeling with lots of other temperaments, too. Programmers in general don't tolerate boredom very well.

Not taking that last step to "clean up" is a curse of our profession, leaving many programs in a less-than wholesome state. Unclean, unfinished programs are the source of many maintenance problems, and of many errors shipped to customers.

Of course, this classification is only a rough model of non-finishers. Many good programmers of all temperaments are excellent finishers, having taught themselves to be aware of their tendencies and counter them with a variety of tactics. Primary among those tactics if the technical review by peers (which is an integral part of the Agile approach but  by no means is not confined to Agile).

So, you offer your “finished” project to your peers for review, and if they agree that it’s finished, you’ve truly gotten something “done.”

If they don’t agree that your work is done, they will give you a list of issues that you need to address before you’re “done.” You then go back to work and address these issues, then resubmit the newest version to another technical review.

Finally, you iterate in this reviewing process until your work passes the review. Then you know you’ve gotten something done.

For more detail on the review process, see, 

Handbook of Technical Reviews, 4th edition

My most challenging experience as a software developer

Here is my detailed answer to the question, "What is the most challenging experience you encountered as a software developer?:

We were developing the tracking system for Project Mercury, to put a person in space and bring them back alive. The “back alive” was the challenging part, but not the only one. Some other challenges were as follows:

- The system was based on a world-wide network of fairly unreliable teletype connections. 

- We had to determine the touchdown in the Pacific to within a small radius, which meant we needed accurate and perfectly synchronized clocks on the computer and space capsule.

- We also needed to knew exactly where our tracking stations were, but it turned out nobody knew where Australia's two stations were with sufficient precision. We had to create an entire sub-project to locate Australia.

- We needed information on the launch rocket, but because it was also a military rocket, that information was classified. We eventually found a way to work around that.

- Our computers were a pair of IBM 7090s, plus a 709 at a critical station in Bermuda. In those days, the computers were not built for on-line real-time work. For instance, there was no standard interrupt clock. We actually built our own for the Bermuda machine.

- Also, there were no disk drives yet, so everything had to be based on a tape drive system, but the tape drives were not sufficiently reliable for our specs. We beat this problem by building software error-correcting codes into the tape drive system.

We worked our way through all these problems and many more smaller ones, but the most challenging problem was the “back alive” requirement. Once we had the hardware and network reliability up to snuff, we still had the problem of software errors. To counter this problem, we created a special test group, something that had never been done before. Then we set a standard that any error detected by the test group and not explicitly corrected would stop any launch.

Our tests revealed that the system could crash for unknown reasons at random times, so it would be unable to bring down the astronaut safely at a known location. When the crash occurred in testing, the two on-line printers simultaneously printed a 120-character of random garbage. The line was identical on the two printers, indicating that this was not some kind of machine error on one of the 7090s. It could have been a hardware design error or a coding error. We had to investigate both possibilities, but the second possibility was far more likely.

We struggled to track down the source of the crash, but after a fruitless month, the project manager wanted to drop it as a “random event.” We all knew it wasn’t random, but he didn’t want to be accused of delaying the first launch.

To us, however, it was endangering the life of the astronaut, so we pleaded for time to continue trying to pinpoint the fault. “We should think more about this,” we said, to which he replied (standing under an IBM THINK sign), “Thinking is a luxury we can no longer afford.”

We believed (and still believe) that thinking is not a luxury for software developers, so we went underground. After much hard work, Marilyn pinpointed the fault and we corrected it just before the first launch. We may have saved an astronaut’s life, but we’ll never get any credit for it.

Moral: We may think that hardware and software errors are challenging, but nothing matches the difficulty of confronting human errors—especially when those humans are managers willing to hide errors in order to make schedules.

https://leanpub.com/errors

Must There Always Be Inferior Code?

Some people claim that when you learn high software standards you will never again develop in inferior ways. Is this true?

I think you can arrive at a meaningful answer by using an analogy:

Some people claim that when you learn high medical standards, a doctor or nurse will never again treat a patient in inferior ways. Is that true?

Seen in this light, the answer is obvious. Most doctors and nurses will not treat patients in inferior ways—unless it's an emergency, like an explosion or a fire in which many people need saving in a hurry. If that happens, the doctor or nurse will return to those patients when the emergency has calmed down. Same in software.

But there do exist a few medical professionals who don’t live up to such high standards. They are, after all, human beings. Yet in spite of their inferior practices, some of their patients do get better. Why? Because humans have built-in healing mechanisms—but software does not.

Software with sick code doesn’t heal itself. Those programmers who develop in inferior ways will eventually produce troublesome code. But the key word is "eventually."

The inferior programmer may not be around any longer when the code's trouble makes itself known, so some inferior programmers can get away with hacky ways for an entire career.

It’s a good manager's job to recognize these inferior programmers and replace them and their code before the true costs of their inferior work become evident.

Some managers overuse the tactic of forcing programmers to code in a hurry, as if there's always an emergency. Just as in medicine, emergency treatment of code tends to produce inferior results. Managers who care only about the short-term will not do anything about their inferior programmers, but they, too, may move out before the consequences of their inferior management become apparent.

That’s why inferior programming practices persist. And, as long as programmers and managers are human, inferior practices will always persist. But they don't have to persist in your world. It's up to you. \

Code in haste, debug forever.

Improving as an Agile Developer

The question was posed: "I'm on an Agile team, and I'm undoubtedly the least experienced member. What could I do to improve my software development skills?" Here's what I answered:

There are many things you can do to improve, and I’m sure you’ll get some good answers. If your Agile team is worthy of its name, you will certainly improve by working with them, reviewing their code, and having them review yours.

From my point of view, however, there is one thing to do that underlies all the other suggestions, and especially learning from your teammates:

Never pretend you know what you don’t know.

Always be ready to ask for help in learning new things.

You may think you have to look extra smart and experienced to impress your teammates, but it’s quite the opposite. Just show your willingness and eagerness to learn. Besides, your Agile teammates will always figure out if you're faking.

Some people might not value this approach, but they’re not people you want to associate with. They’re probably pretenders themselves. They're certainly not good Agilists.

How do I get better at writing code?

Nobody writes perfect code. Anyone, no matter how experienced, can improve. So, you ask, how do I get better at writing code?

Of course, to get better at writing code, you must practice writing code. That much is obvious. Still, just writing the same poor code over and over, you're not likely to improve by much.

Writing is a different skill from reading, but reading code is necessary if you want to improve your writing. As with writing natural language, you build up your skill and confidence by reading—and not just reading your own output. So, find yourself some examples of good, clear code and read, read, read until you understand each piece.

Be careful, though. There’s lots of terrible code around. You can read terrible code, of course, and learn to analyze why it’s terrible, but your first attention should be on good code. Excellent code, if possible.

Where can you find good code? Textbooks are an easy choice, but be wary of textbooks. Kernihan and Plauger, in their book, The Elements of Programming Style, showed us how awful textbook code can be. Their little book can teach you a lot about recognizing bad code.

But bad code isn't enough. Knowing what's bad doesn't necessarily teach you what's good. Some open source code is rather good, and it’s easy to obtain, though it may be too complex for a beginning. Complex code can easily be bad code.

Hopefully, you will participate in code reviews, where you can see lots of code and hear various opinions on what’s good and what’s less than good.

Definitely ask you fellow programmers to share code with you, though beware: not all of it will be good examples. Be sure the partners you choose are able to listen objectively to feedback about any smelly code they show you.

If you work alone, use the internet to find some programming pen pals.

As you learn to discern the difference between good and poor code, you can use this discernment in your reading. After a while, you’ll be ready to start writing simple code, then work your way up to more complex tasks—all good.

And date and save all your code-writing examples, so you can review your progress from time to time.

Good luck, and happy learning!

Classifying Errors

I received an email the other day from Giorgio Valoti in Italy, but when I wrote a response, it bounced back with "recipient unknown." It may have been a transient error, but it made me think that others besides Giorgio might be interested in discussing the issue of classifying errors, so I'll put my answer here and hope Giorgio will see it.

Here's the letter: 

Dear Mr. Weinberg,
My problem is that I’m looking for good way — maybe a standard, more likely a set of guidelines — to classify and put a some kind of label on software defects.

Is there such a thing? Does it even make sense trying to classify software defects?

And here's my reply:

https://leanpub.com/errors

Hello, Giorgio

It can certainly make sense to classify errors/defects, but there are many ways to classify, depending on what you're trying to accomplish. So, that's where you start, by answering "What's my purpose in classifying?"

For instance, here are a few ways my clients have classified errors, and for what purposes:

- cost to fix: to estimate future costs

- costs to customers: to estimate impact on product sales, or product market penetration

- place of origin in the development cycle: to decide where to concentrate quality efforts

- type of activity that led to the error: to improve the training of developers

- type of activity that led to detecting the error: to improve the training of testers

- number of places that had to be fixed to correct the error: to estimate the quality of the design

- and so on and on

I hope this helps ... and thanks for asking.

--------------end of letter-----------

As the letter says, there are numerous ways to classify errors, so I think my readers would love to read about some other ways from other readers. Care to comment?

Should I learn C++ or Python?

When I first saw this question on Quora, there were already 47 answers, pretty much all of them wrong. But the number of different answers tells you something: choice of programming language is more of a religious question than a technical one. The fact is that if you want to be a professional programmer, you should learn both—and at the same time.

When we teach programming, we always teach at least two languages at the same time, in parallel. Assignments must be done in both (or more) languages, submitted along with a short essay on why the solutions are different, and why the same. That’s the way to develop some wisdom and maturity in the coding part of your professional work.

Some of the respondents asserted that programming languages are tools. If that’s an appropriate metaphor, then how would you answer this question of a wannabe carpenter:

"Should I learn saws or screwdrivers?"

Do you think someone could be a top-flight carpenter knowing only one?

So, stay out of this quasi-religious controversy, which can never be settled. Instead, spend your valuable time learning as many different programming languages as possible, at least 5 or 6. You won’t necessarily use all of them, but knowing their different approaches will put you far above those dullards who say:

“I only know Language X, but it I still think it’s the best language in the world.”

See also: Understanding the Professional Programmer 

and The Psychology of Computer Programming