Software Development: Experience vs. Performance

Experience vs. Performance

"Experience is the best teacher."

"Experience is the only teacher."

"Experience keeps a dear school, but fools will learn in no other."

"Experience doesn't necessarily teach anything."

"Experience is the only school in which the test comes before the lesson."

There may be more folk wisdom on the subject of experience than on the subject of love. After all, if we couldn't appeal to experience, how could we old roosters keep control of the henhouse?

When I was young, I was frustrated by lack of "experience" at every turn. No matter how hard I worked, I could gain no more than one year's experience each year. At 19, a year is a lot longer than at year 79.

"Experience" is also proving frustrating to many programming managers, whatever their ages.

"What is the value," I'm often asked, "of one year of programming experience?"

"Does productivity actually increase after the third year of programming experience?" "Don't programmers 'burn out' after five years?"

Quite a few programming managers, if they were asked to plot the productivity value of experience, would produce a graph something like that in Figure 1. They've found that after roughly three years, additional years of experience don't seem to add significantly to a programmer's productivity.

 Figure 1.

Managers who hold this model naturally are unwilling to pay a premium for many years of experience. Usually, they will seek programmers on the job market with one or two years of experience.

One problem with this model is that it may fail to consider problem difficulty. In most shops, the more experienced programmers are given, on the average, more difficult programs to write. If the measures of productivity fail to consider problem difficulty, the more experienced programmers will naturally have their productivity underestimated. In some cases, a more realistic model would be that of Figure 2.

Figure 2.

This slightly less pessimistic curve is supported by the results of a number of studies in the psychology of programming. When programmers of varying experience are asked to perform the same task—coding, finding bugs, developing test data—the average performance versus experience curve frequently looks like that of Figure 2.

Even so, a manager studying Figure 2 may still wish to follow the strategy of hiring programmers with very little experience—in order to take advantage of the steeply rising segment of the curve.

Managers in smaller companies, however, may have more discretion. If they are perceptive, they can profit by noting the difference between each individual programmer and the presumed average.

The same studies of programming support the view that a manager does well by appraising the quality of each candidate's experience. Does the programmer really have "10 years of experience," or is it more accurately characterized as "one year of experience, 10 times?"

If we divide the programmers in each study into two groups—high performance and low performance—we generally find that the experience curves look like Figure 3.

Figure 3.

The curve we see in Figure 1 or Figure 2 is, probably, a mixing of these two curves into a single curve.

Conversely, programmers often tolerate low-paying jobs their first year or two, in order to be in a company that pays well for longevity, rather than for actual output.

Both Figure 1 and Figure 2 represent models of the average programmer in really large companies, employing hundreds or thousands of programmers, personnel policies may force them to follow the averages in hiring and setting wages.

Do these two curves represent anything real, or are they artificial manipulations of data? My own experience with,such studies and on the job tells me that there is an important truth here. In my next blog entry, I'll examine this truth and why so many managers fail to understand it.