How We Used to Do Unit Testing

Reader Charles sends a question I've frequently had when consulting with testing organizations who are weary of having code thrown at them "over the wall."

Charles

From your experience in software, what are some good practices you have found for what is now known as "Unit Test?"

I have mine, I want to see if we have similar experiences.

I also have a motive, I get the opportunity to write our "Unit Test Procedure", so I want to capture "Good Practices" that developers can use.

Ack!--Unit Test Procedure!!! I wish we could have names like, "Menu of Good Unit Test Practices, Choose One or More that Make Sense for your Unit, or Use and Add a New Practice to the Menu of Good Practices (Include Provenance or Example)."

Jerry

Honor Unit Testing

1. Well, that's probably number one, the name and prestige you give to the process. If you belittle it, you will have poor results. If you reward it ... finish the sentence. I don't recall if we even had a name for it in the early days. It was just assumed that any pro would do a damn good job of this activity, not to be embarrassed by unit bugs found in integration test or system test--or god help us, in production.

Build Small and Simple

2. The next most important thing is to design to build in testable pieces. Small as possible, but no smaller. Cleanly structured, with no testing traps like hidden memory that might make the code not really reentrant.

Be Completely Open

3. Next is complete openness, getting as many eyes on it as you could get, but certainly not just one pair.

Test First

4. Today what we did is called Test Before Code, or perhaps Test Before Design. Ideally, we sketched out a set of test cases before putting pencil to coding pad. (We didn't have terminals or PCs in those early days.) These were punched into cards and put in the permanent test case library.

Take Advantage of Individual Skills and Help Each Other

5. After that, I recall things breaking down into tactics favored by each individual. Generally, we unit tested each other's code, and anyone having trouble could and did call on anyone else to help out.

Don't Green Dot

I think if people did these things today, we'd be in a lot less trouble. I know because there are people who do these things today, and they tend to stay out of trouble. On the other hand, people who just throw code at a compile and say "it's unit tested" as soon as they see a green dot, tend to be in trouble all the time.

Comments welcome!

What is an Alcoholic?

A commenter describes his drinking habits and says they don't interfere with work. He wants to know how I define "alcoholic"?

I think he and I agree that the question turns on how your drinking affects your work, short term and long term. Since everybody's physiology is unique, I don't think the definition can depend on how much someone drinks. What would put me in the hospital (I have a severe reaction to alcohol), might just be a thirst-quencher for someone else.

One problem with this approach is that alcohol definitely tends to take the edge off one's judgment. I have dealt with a number of worker/drinkers who believe their work is not affected by their drinking--but the reason I was asked to speak with them is that their work effectiveness has been dropping noticeably, according to their managers.

I see lots of moral judging about alcohol consumption, which in turn leads to a lot of defensiveness on the part of those who enjoy drinking. For me, I don't care if your work is affected by alcohol, M&Ms, or listening to too much opera. If something is affecting your work, then that's an issue for your manager and possibly your co-workers.

If it's affecting your health (long or short term), that's your business, and perhaps the business of your family. Of course, if it's affecting your driving, then it's my business as long as we're sharing a road. I won't accept rides from people who have been drinking--or who might be otherwise impaired.

In short, I'm concerned about your alcohol habits only insofar as they affect me. It's an individual judgment, not a stereotype--but it's definitely a sterotype for lots of people, something alcoholics have to live with in today's society.

The Sense of Smell

I've been in less than tip-top condition lately, so you haven't seen any posts here for a long time. I just don't have the wherewithal to compose new stuff that's not whiney, so I think I'll start posting some really worthwhile stuff when it comes in from my correspondents (with their permission, of course).

The following is from Jim Batterson, who is currently teaching classes in China. Here's his words:

---------------------------------------------

I thought of a story and a half and I wanted to share it with someone, and you (Jerry) came to mind.

Let's begin with a simple experiment that I remember from a psychology class years ago. I guy thinks he is one of eight test subjects in an experiment. They all sit on a panel and he is number seven. The exercise is simple. They are shown two lines and asked to say which line is longer, A or B. B is clearly longer, but the lines are at different angles and such. One through six all say A is longer and our poor subject goes along with the crowd and agrees with them. Not clear whether he is appeasing or whether he doubts his own judgement or what. This repeats many times. Of course, one through six are shills planted by the experimenter.


Scenario two: same as the first except number two gives the right answer. All the support our guy needs to give the right answer every time.


So there I was in New Jersey sitting around a table for a project meeting. I was only tangentially on the project. I maintained the system that was being partially replaced, and had written an extract program that fed data to the system under development. When I sent them a test file they were supposed to run it through their edits and the rest of the system and I got some feedback at the end, but I really wasn't getting anything meaningful back.


Nevertheless, they went around the room and everyone seemed to be reporting that all systems were go, ready to fly in about a month. They'd been working on it for over a year. I, too, had done everything I was supposed to do and could have reported the same, but the manager detected a bit of hesitation in my voice (not very subtle) and asked me what I thought. She was a pretty sharp woman. I wasn't sure what to say, because I really was hardly involved in the project at all.


I told her that I had been on a lot of projects before, which she already knew. I didn't have any facts or data or even examples to point to, but "this project doesn't smell like a project that is a month away from going live."


I think that her senses, too, were telling her the same thing, but she needed another voice to confirm her suspicions. My sense of smell was all the confirmation she needed to drop the delivery date and push things back three months. She was still off by a month, but the system went live five months later.

Jim

Jerry Weinberg's New Website

At long last, my new website design is up and running(?). Same url, http://www.geraldmweinberg.com, but some of the links to individual pages may no longer be valid because of the new design. Many thanks to Pati Nagle for design advice.

I welcome feedback on any aspect of the site. New material and links will be added incrementally.

The Consultant's Money-Back Guarantee

On LinkIn, recently there was a question about what a client had to do to hire a consultant who wouldn't rip them off, just taking money for no measurable result. In my response to the question, I wrote:

I make this easy for my clients by two principles of my consulting business:

1. At the end of every consulting visit, I ask them to evaluate the worth of my contribution. If it's not worth more than they paid me, we either adjust what I'm doing or we terminate the relationship.

2. If they don't feel what I've done is worth what they've paid, they can have their money back, no questions asked. I make sure they know this up front--though I've never had to give back their money.

If a consultant doesn't give you both these things, don't hire them.

Why Give Their Money Back?

Pradeep Soundararajan wrote, in response:

While anything that any human does is a heuristic, why should a consultant want to give back the money for the client not happy with what the consultant influenced?

For instance, I go in as a Consultant to solve a problem. At the end I help the client understand that there is more investment he ought to do in order to ship the product. The client feels unhappy because he hired me thinking that I would help him ship the product with existing budget.

Maybe what I did is what other good consultants might have done, so should I return his money back because he feels disappointed?

Why? Because it's your job to satisfy the client. That's what they pay you for. If you buy a laptop from Apple or Dell or HP and when you try to use it, it doesn't work, don't you expect to get your money back? If you don't, would you ever buy a computer from Apple or Dell or HP again? So, one reason to offer a money-back guarantee is to develop future business with that client.

Another reason is to keep that client from spreading bad news about you to 16 other potential clients. (That's what people do when they feel they were cheated by a vendor. When they feel they got a good value, they tend to tell only three other people.)

The Steering Wheel and Brakes

For my consulting assignments, this guarantee acts like a combination of steering wheel and brakes. It guarantees that the client and I will pause periodically and see if we're going in the right direction. If we're not, we can use the steering wheel to change course, or in extreme cases, just apply the brakes and end the assignment.

Would you drive a car without a steering wheel and brakes? Then why would you want to take on a consulting assignment without them?

I guess maybe you wouldn't want them if all you wanted was a runaway car, one that somehow kept paying you as it careened away toward a fatal crash. If cash flow is the only reason someone is in the consulting business, I hope they hit that wall as soon as possible. People like that give my profession a bad name, and thereby hurt a great many ethical and effective consultants.

Don't get me wrong. I think money is a fine reason for being a consultant, but not if it's the only reason. I want to get paid for my work, but only if I'm actually helping people. I don't want to be a fraud.

What Is Value?

But perhaps the most important reason to offer this guarantee is to force them, and you, to think about what value means to each of you.

Pradeep goes on to say: I think such traps have to be cleared before a Consultant gets in by asking questions like, "What value means to you?" and "Can you think of things that you'd be disappointed to know at the end of this assignment?" and "Do you have insight about any kind of information that could cause you nightmares?"

Well, yes, of course you want to clear this up before you go in, but that's not always possible. People often don't know in advance what they want. They only know when they see it. Many of my clients ask for one thing when I come in, but through my consulting learn that they really wanted something else that was more valuable.

To take an extreme case, at one client, the man who invited me in to help with strategic planning learned that his alcoholism was tearing his own organization apart, and what he needed to do was fire himself and go into treatment. That wasn't what he asked for at the beginning of the assignment, but by the end, he knew that this result was far, far more valuable. I asked him if he wanted his money back (he was part owner of the company) and he said no. In fact, he wanted to pay me more--but I refused. I did my job (helping his company improve), and I earned my pay. That was enough for me.

What If Every Consultant Did This?

Pradeep then says:

Not every consultant would want to return back the money and that could be a good marketing stint or building credibility or a distinguishing factor but if every consultant does that ( which is not an easy thing ) then it might end up causing confusion of whom to pick.

Pradeep is right. It's not an easy thing. In fact, it's so unlikely that it's not worth worrying about.

But, if a lot of consultants used these principles, maybe there wouldn't be so many derogatory jokes about our profession of consulting.

Intelligence Isn't Enough

Pradeep then concludes:

An intelligent client picks an intelligent consultant and that's a heuristic, too.

Well, yes, that's certainly a heuristic, and a necessary one. But not by any means a sufficient one. Intelligence without ethics is like a biological weapon in the hands of a terrorist. Very powerful, to be sure, but you'd much rather not have it on your premises.

How Can You Recognize Alcoholism in a Service Provider?

Jeff wrote: "Do you have a specific test for that [alcoholism in service providers]? Many of the alcoholics I've known hid their problem well, at least for some period of time."

Good question, and one I couldn't answer at the time. After being placed in jeopardy with the government as a result of Provider's actions, I began to study the problem with great interest. Here are some signs I now recognize that I didn't pay attention to at the time:

Late for Appointments and Missing Deadlines

This one I definitely noticed, but didn't recognize the possible significance. I just told myself that Provider was a person who was "habitually late." Like many of the other signs, lateness could be attributed to many things besides alcoholism, so I let it pass without comment.

Depression and Mood Fluctuations

Again, I noticed this, but didn't appreciate the possible significance. I believe I thought, "Well, such providers aren't the most sparkling of personalities." Actually, my next provider proved even that assumption wrong. She's terrific.

Mistakes

Everybody makes mistakes, and I tend to be pretty generous in allowing for them. Some of Provider's mistakes were hidden, and that was my fault for not having a reasonable feedback mechanism. But I had noticed a rather higher level of mistakes than I'd like to see in a provider, and I just let it pass.

Personal Problems as Excuses for Mistakes and Lateness

Provider was never short of excuses for mistakes and lateness. Health, problems at home, "the dog ate my calendar"--he was very creative.

Choosing Lunch Dates in Drinking Places

I didn't have lunch with Accountant very often (taking lunch alone may be another sign), but looking back, I realize that he always insisted on restaurants that had a bar.

Showing Up Intoxicated

I never noticed this with Provider, but in subsequent years, I've noticed it with other service people. Whether they're alcoholics or not, this is unacceptable. For example, someone operating a power lawnmower when drinking is a risk to his life and limb--and to my entire business if he sues me for cutting off his foot while working for me.

Health Problems

Anybody can have health problems--I'm a prime example. But someone who is consistently coming down with one misery after another might be showing symptoms typical of alcoholics. Same is true for frequent injuries and accidents. But, of course, they might just be a natural klutz.

Speaking affectionately about Drinking

I should have recognized this one, for my mother was an alcoholic. She often spoke lovingly about her Southern Comfort. Provider's drink of choice was different, but he seemed to have the same love affair. Affairs, really. He loved 'em all.

Signs, not Proof

None of these signs prove that someone is an alcoholic. They could be signs of other things--other addictions or something quite innocent. But my job is not to prove some provider is an alcoholic, which can be incredibly difficult. Alcoholics are experts at denial, rationalization, dreaming up excuses, blaming others, manipulating you, or hooking into your caretaker needs. Besides, their alcoholism is none of your business.

Your Responsibility

What is your business is your business. You hire a provider to do a particular service. If they don't do that service well enough, it's your responsibility to replace them, not to make excuses for them. And especially not to fix them. Set performance criteria. Communicate those criteria. Observe performance relative to those criteria, and take action when performance doesn't measure up. Why it doesn't measure up is not your job.

I didn't do those things with Provider, so I got snagged into his drinking problem. It was his problem, but it was my responsibility to protect myself. I now do a better job of fulfilling my responsibilities as a business owner. Overall, I've protected myself not just from alcoholism, but from other problems that are not my problems.

But She's My Friend

Does this sound heartless and cold? Maybe you're good friends with your service provider? Can you treat your friends like this?

To take just one example, I had a copy editor who had trouble getting to work in a timely manner. We have flexible working hours, but I couldn't depend on her for any schedule. Turns out, she was not an alcoholic, but was depressed over her mother's death three years earlier. She tended to sleep 12 or 14 hours a day. After causing me to miss an important mailing deadline one afternoon, she said, "Oh, if you need me and I'm not here, you can just call me and wake me up."

Not my job. Not as her client. I replaced her with an editor who could wake herself up.

Then, as a friend, not a client, I helped the copy editor find a really good therapist. Just as I helped Provider fulfill his AA twelve steps. It turns out, if you want to help people with such personal problems, it's easier if you're not hiring them to do a job.

Services for a Small Consulting Business

I haven't blogged for almost two months. I've been too busy. Busy is usually good for a consulting business, but not this time. A month or so ago, I parked by our UPS store and headed inside to pick up our mail. At least I thought I was picking up mail until I saw the sign on the door: CLOSED UNTIL FURTHER NOTICE.

Closed? All my mail was inside, and there was no other information. A kindly old gentleman must have seen the befuddled expression on my face and came out of the golf store to inform me that the New Mexico tax people had come that morning and sealed up the UPS store for non-payment of taxes.

I will spare you the pain we experienced as a result of this failure of one of our service providers–mail delayed two months or more, or lost; literally hundreds of change-of-address notices sent, checks from clients that we couldn't get our hands on; dunning notices from vendors whose bills we hadn't received; and who knows how many mails lost forever. I want to tell you about the benefits, instead–lemonade from lemons.

The principal benefit from this potential disaster was making us stop and review all the vendors we counted on.

Communication

- We use the mail drop because the US Postal Service does not deliver mail to our house or office. Moreover, they do not hold the quantity of mail we accumulate when we are working out of town for a couple of weeks. We also use them as our fax station so we don't have to bother with a fax machine in our office. Same for copy machines.

- Same for wrapping and mailing. When I had my own publishing company, we had a mail room with two full time employees. When business got so good I was about to hire a third, I decided that managing them was a drag on my consulting time, so we sold the publishing business.

- After careful analysis, we chose a new mail drop–another UPS store–which seemed to be on a much more solid financial basis, and about the same distance from our house. (Going there has introduced us to some new restaurants in the locality, which is a second benefit.)

- We also use voice mail. I do so obviously because I'm often away from the office, but even more because I won't allow myself to be interrupted by any idiot with a quarter. If a call is important, I'll get it eventually on the voice mail. For many years, I had an administrative assistant to answer my phone and perform other services, but an employee creates a fixed cost that is not easily reduced. And, it's a person to manager. I don't get paid for being a manager, but only for being a consultant.

Computers

- Some of my techie consultant friends maintain their own servers and other networking equipment. They also host their own websites. These activities are so much for for us nerds, they can be a trap. For a few dollars a month, I can have full-time professionals doing those things for me. What I care about is their reliability, so I don't have to spend time looking over their shoulders.

- Curiously, I do my own bookkeeping. I used to hire a bookkeeper, but I found I was spending as much time preparing papers for him as I needed to handle the paperwork from end-to-end myself. More than that, with someone else handling the books, I found I was losing track of the way my consulting business was actually running. Follow the money if you want to understand your own organization.

Taxes, Legalities, Insurance, Banking

- On the other hand, I would never try to do my own taxes. Bookkeeping hasn't changed much since I was in high school, and what changes there have been have made things easier (with computers). But keeping up with the tax code, that's another matter entirely. I figure my accountant saves me thousands of dollars every tax season. If she didn't, I'd be looking for a new accountant. She's been my accountant now for more than twenty years–ever since I fired my previous accountant for failing to file my tax returns for the several years he was in an alcoholic fog. Now, I don't hire service people who have drinking problems.

- I retain several attorneys, not on retainers, but on an as-needed basis. When someone plagiarizes my writing, I need an intellectual property specialist, but that happens only about once every ten years (so far). When I buy or sell a building, I need a different kind of attorney, as I do when someone accuses me of trespassing when I'm hiking on a trail in the National Forest. In the past, I had one general counsel for all my legal needs, but it's really impossible to find one attorney who is expert in all areas. In general, though, I keep my attorney needs to a minimum. For instance, all my client contracts are simple letters based on mutual trust (except we always put dollar amounts in writing.

- Insurance, to me, is just another form of legal matter. I use an agent I trust, and I don't look for the cheapest policies. What I want is real insurance–policies that protect me from catastrophic financial loss, not policies that pay me $100 when someone dents my car.

- I do use a bank, rather than keeping my money in a mattress. Actually, I use two banks, so my eggs are in separate baskets. Every so often, one of the banks starts doing "creative banking," so I switch to another one. Since I have two, I can switch painlessly without interrupting my business. Banks today are pretty much a commodity business. If they hassle me, or have poor security, or lack convenience, I simply switch.

Physical Services

- Janitorial work, we farm out, too. We're rather neat, but every two weeks we have a professional crew in to do the big cleaning. But I clean up after the new puppy.

- Nobody can really farm out security, though we do have an alarm company. Our major physical security is our crew of trained German Shepherd Dogs roaming our fenced property. People don't generally bother us, even to waste our time trying to sell us magazine subscriptions or contributions to their one true church. Computer security, I take care of myself. No need to go into details here, but the fewer people who know how you secure your computers and data, the better off you are.

Editorial Services

- Finally, as writing is a huge part of our business, we retain professional editorial help on an as-needed basis. Part of that is using old-fashioned publishers, who still value the work of real editors, and who turn around manuscripts promptly as promised.

Our Philosophy

So perhaps you can see our philosophy, which may be useful for other small consulting firms:

- Fundamentally, our billing rate is far higher than any of the services we use, so it doesn't pay to perform these services ourselves. Better to spend our time earning money.

- But flexibility is important, too, because there are dry spells where there's no paid work at hand. In the past, that situation was much more common,d so we always structured our work so we could take over most services if cost-cutting was the word of the day. In the end, though, you can't make a living by cutting costs.

- Since our time is so valuable, we shed a service if it requires too much involvement of our own. We like to hire small businesses where we know the owners and can count on them to resolve problems quickly. They understand our situation as a small business. And, they don't run in the second or third string on us, the way some of the larger firms do. We know the people who work for us, and we deal with them as individuals, the way we like to be dealt with ourselves.