Estimating Projects: A History of Failure

When I lived in Crested Butte, Colorado, I sometimes hiked across the mountains to Aspen. There were five hiking trails that I knew of, all of them former toll roads from the old gold-mining days when the only way to reach Aspen was through Crested Butte. As I struggled to breathe going up these 13,000+ foot passes, I often wondered why there were five toll roads. But the time I'd hiked all five, I knew the answer. There were five toll roads because none of them were very good.

Consultants are frequently given the task of estimating, or helping to estimate, projects. To start with, they're asked to estimate how long it's going to take to solve their client's problem—which is usually undefined. Although there have been billions of words written on dozens of methods of estimating projects, most of them are, in my opinion, useless or nonsensical. Or both. Like toll roads to Aspen, there are so many because none of them are very good.

Why do I say they're not very good? Well, why were the toll roads to Apsen not very good? Mostly, I think, because if you're walking from Crested Butte to Aspen, you have to cross an extremely high barrier. We might want to be able to hike over a 13,000 foot pass without breathing hard, but it's impossible (at least for most of us).

And that's the primary reason that all these estimating methods are not very good is exactly the same: they are trying to do something that's impossible—predict the future. Sure, we'd all like to do this, but we can't.

People have been trying to predict the future as long as people have been around, so I thought it would be useful to read about methods from the past, so perhaps you could relate them to some of the failures of today. The following post is from dgc, a friend who has worked for a long time in the software industry, mostly as a contractor and consultant—long enough not to take himself too seriously, a good lesson for all of us consultants.

Some Estimating Methods

The fabulous Wizard of Oz
Retired from his racket because,
What with up-to-date science,
To most of his clients
He wasn’t the Wizard he was.
- Anonymous

Since estimation seems to have much in common with the mantic arts (i.e., the arts of divination), I turn to the writings of an expert in that field: Dreamas Gentilharte Cheynelokk, a nearly thousand-year-old wizard who writes about the mysteries of CMMI (Comprehensive Magistrate of Magical Integrity) certification.

For those unfamiliar with this form of CMMI, Dreamas briefly describes its five levels of magical competence as follows (the translation of this text is by Dr. Devin G. Kettenschloss in "Paduan Vellums, Volume VI, Item lvii- A Translation" following the seminal work of the late Dr. Blanche V. Foote-Falles):

"(1) Erotic [1]: animalistic, sensual, like kindling consumed in a flame, uncontrollable, never twice the same, exciting

(2) Telestic [2]: ritualistic, repeated, like an unbending oak planted in the earth, formal, delimited and limited capacities, routine

(3) Mantic [3]: prophetic, governable, like a ship sailing home on uncharted seas, divinatory, observed and proven by trial, directed

(4) Poetic: rhapsodic, meted, like a falcon riding steady the rolling air [4], quantifiable, sound and proportioned in structure, balanced

(5) Ecstatic [5]: euphoric, evolving, like the shining cosmic quintessence [6], heavenly transcendence married to earthly immanence, sustained."

Dreamas goes on to describe hundreds of mantic techniques useful across all aspects of software development. But for the topic of estimation, the following description of the art of "Alectryomancy" seems relevant when speaking about making time/schedule estimates for a project, especially since the technique used differs based on the CMMI level of the organization (again quoting from the previously cited translation).

"Alectryomancy [7]: Divination by the Actions of Poultry Pecking at Grain

So much software development goes awry for lack of sufficient numbers of dedicated top-breed poultry!

For those functioning at the CMMI Level 1 (Erotic), any hen or rooster of any kind will do. Simply put out a handful of grain and let the chicken loose for one minute. At the end of one minute, guess how many grains the chicken ate and multiply by 3. This calculation will yield the needed number of hours, days, weeks, months or years it will take to complete the project.

Those groups at CMMI Level 2 (Telestic) laugh at that naiveté. The wizards at this level realize it is not that easy. First, more chickens are needed, at least three but never more than nine. Second, before being let loose on the grain the chickens properly motivated with loud chanting and the waving of gleaming, sharpened axes. Third, the grain for each bird is placed in a line running east to west and the first grain eaten by each bird is the estimate. That is, if the 4th grain from the east is the first eaten by a hen, then the estimate from that hen is 4. Finally, after all chickens are done, the lowest estimate is picked with any multiplicative factor determined by the wizards using Capnomancy (divination by smoke) or Catoptromancy (divination by mirrors placed under water).

But groups at CMMI Level 3 (Mantic) realize estimation is a daring, daunting and dangerous undertaking. So in their practice, they only use roosters. Each rooster is allowed to make its estimate by pecking a grain in its own sealed off arena. A variety of techniques for deriving the numerical estimate can be employed (counting grains, counting volume, which grain is first picked, which is last picked, etc.) with skilled wizards selecting in advance the appropriate technique by means of Gyromancy (by whirling until dizzy). After each rooster has produced his estimate, all the roosters are placed in one arena for a cockfight {SIDE NOTE FROM 'dgc': I realize the cruelty of this technique may offend some people's sensibilities, but that is what the original text says} with the winning cock's estimate being adopted [8] and everyone being held to that estimate under threat of the hatchet.

Groups at CMMI Level 4 (Poetic) with more refined sensibilities bemoan the waste of possibly good estimators by the Mantic level wizards as well as the blatant discrimination against hens. Therefore, they eschew the wastefulness of cockfights and use both hens and roosters; but they also carefully record and plot age, breed, and accuracy of auguries over time. In addition, no matter what anyone says about chickens and lips, they firmly believe in the importance of using Labiomancy (divination by lip reading) as part of their process.

Finally, wizards at CMMI Level 5 (Ecstatic) realize that the work of estimation is never done. So they use the greatest number of poultry and have them estimating and reestimating often using dozens of techniques and inventing new ones whenever needed. At this level, the chicken coop and yard is a constant bustle of activity yielding untold numbers of eggs. After all, in the end, it is all about the real egg production."


Footnotes (From the translation)

[1] From the Greek meaning "sexual love" as associated with Eros, who is not the cute cupid of modern times but is rather the passionate young god, son of Venus, as best described in Apuleius' story of Psyche and Eros in "The Golden Ass" (Chapters 7-9).

[2] Very rare in modern English. Derives from the Greek "telestes," meaning a "hierophant or expounder of sacred mysteries."

[3] The Romans, especially Cicero in his treatise "De Divinatione," clearly preferred the word "divination," which derives from the Latin "divinus" and indicates a deific origin, to Plato's "mantic," which is equivalent to the Latin word "furor" meaning "madness, lunacy, or raving."

[4] Echoing Gerard Manly Hopkins' "The Windhover" (http://www.bartleby.com/122/12.html).

[5] There is a delicious irony in the CMMI master's choice of the word "ecstatic" for their highest and ideal level. From the writings of mystics throughout the ages, ecstasy is often associated with displays of frenzy and agitation bearing strong sexual overtones. See, for example, the writings of the Spanish mystics St. Teresa of Avila ("Interior Castles") or St. John of the Cross ("Dark Night of the Soul"). Given these strong sexual associations, it is possible it might be somewhat difficult to see how the Ecstatic level differs from the lowest level, Erotic.

[6] Here, Dreamas means the ancient and medieval concept of the perfect material of which the stars were made: an exact blend of all other types of matter (fire, earth, water, and air).

[7] An alternate spelling is "Alectromancy."

[8] Note how in many ways this is similar to the Delphi technique of estimation some speak of in software engineering.

Money Questions 1.

Q: What’s the most common question you’re asked since the publication of your two books on consulting?

A: It’s no single question, but a collection of questions around a common theme: money and the consulting contract. In The Secrets of Consulting, I’ve written about how to figure out how much to charge, but not about other issues like the messy business of collecting what you charge, or how to deal with price negotiations. So, for a while, I'll provide “extra chapters” to answer address some of these untidy questions.


Q: A prospective client wants me to lower my rates. He says he can hire other consultants for 30% less than I’ve asked for. What should I do?

A: Don't ever let yourself be a commodity. I learned this lesson early from my grandmother, Ethel, who owned a grocery store. “Don't be a commodity,” she always said. “Everyone can buy the same groceries, and the supermarkets can buy them cheaper than I ever can. So I have to offer something different.”

Ethel did things in the store to differentiate her from other stores in the neighborhood. She was nicer to her customers. She knew everyone one of them by name, and gave each one of them personal attention. She loved to take special orders from them, and went to a great deal of trouble to find exactly what they wanted. In those items, like meat and other perishables, she offered better quality than the supermarkets.

She offered many “extra” services, some of which have finally been taken over by today’s supermarkets. She offered credit, which was rare in grocery stores in those days, and absolutely taboo in supermarkets. She cashed checks, put things away for her customers who called, and provided a delivery service (which was often me, when I was in town).

In general, she was aware of her customers' requirements, and responded with inventory that matched their individual tastes. (examples)

So, offer something extra, but don’t lower your price.

Q: Should I pretend I have more qualifications than I really do, to keep my price up?

A: Don't pretend you're what you aren't just to get a job. If you do, you'll soon be complaining that the job doesn't fit the real you.


Q: My client just won't budge. Aren't there some circumstances in which it's okay to lower my price?

A: If they give you something extra, you can lower your price accordingly. For instance, if they're willing to pay you a non-refundable retainer up front, you can offer a discount. Or perhaps they'll pay certain expenses that you were prepared to pay yourself. Over the years, clients have given my the use of cars and computers as extra compensation—compensation that didn't really cost them anything because the cars and computers were just sitting around not being used.

Perhaps the biggest break I give is when the client does not require I come to their offices to consult or teach. If they come to me, the get a discount. Some young consultants think of the travel itself as a form of compensation, as I did long ago. Now, travel is a definitely negative for me, so I reward those clients who don't require it.

There-then-them vs. Here-now-us

On my writing blog the other day, I posted the following paragraph as part of a writing exercise:

Anger, for a consultant, is a costly luxury, and I am by nature somewhat of a cheapskate. By eliminating there-then-them anger, I cut my angry outbursts in half. By noticing my pattern of anger escalation, I dampen nine-tenths of the remaining half to the point where it doesn't interfere with my consulting practice. That leaves only about 5 per cent of the angry episodes I used to have, just one in twenty. Although this seems a dramatic improvement in frequency, it doesn't result in an equally dramatic improvement in the cost of my angry outbursts.

In response, dgc said.: 'What does "there-then-them" mean in the phrase "there-then-them anger" in the original of the Writing Blog exercise? I can’t quite determine the meaning of that phrase from the context.'

Here's my answer:

There-then-them is in contrast to here-now-us, which are the problem-solving conditions expert consultants try to establish and maintain. When people are responding to something that happened somewhere else (there), or at some other time (then), or with some other people (them), you're not going to have much luck dealing with problems.

In the example above, sometimes I find myself growing angry at my clients, only to realize that I'm responding to something similar the client said or did at another time or place. Or perhaps I'm responding to something similar my mother used to say to me when I was five years old. Clearly, I'm going to have to bring my mind into the present context if I'm to be effective.

And, of course, my clients have the same need for here-now-us, so I often them solve problems by bringing them back to the present context, especially when their emotional reactions seem to be directed at me for no reason I can ascertain. One woman attacked me in a meeting, accusing me of mocking her. I couldn't figure it out until I managed to persuade her to give me details. It turned out that I had used the word "snow," which was the name of her former husband whom she had recently divorced. She thought I knew about the divorce (other people did, but I didn't) and was taunting her by alluding to it. Using here-now-us cleared up an enormous emotional outburst.

So, dcg, does that clear it up?