In 2007 my colleagues at Construx Software and I updated the list of classic mistakes from my 1996 book Rapid Development. Throughout 2007 we conducted a survey to determine the frequency and severity of these classic mistakes. In other words, we wanted to get a more quantitative sense of just how "classic" these classic mistakes are.

More than 500 people responded to the survey. The majority of them were involved with web and business systems. A significant minority were involved in shrink wrap/commercial systems, and about 10% were involved in embedded, system critical, systems, SaaS, or other kinds of software. About half the respondents were in lead/architect roles, about one-quarter in individual technical contributor roles, and the rest were in management or dual management/technical roles. The results are available in a white paper, "Software Development's Classic Mistakes 2008." You will need a login our main web site to download the white paper. (The log in is free.)

Excerpts from the Classic Mistakes Survey

Based on the survey responses, we computed the approximate frequency of the mistakes surveyed. Here is an excerpt from the white paper that shows the approximate frequency of occurrence of the most common classic mistakes:

We also examined how severe the mistakes are when they occur. This excerpt from the white paper describes which mistakes produce Catastrophic or Serious consequences the most often:

Finally, we made an assessment of which classic mistakes are most damaging overall. We multiplied the approximate average frequency of each mistake times its average severity to arrive at a Mistake Exposure Index (MEI). The MEI ranges from 0 to 10, with 10 being the worst. Here is an excerpt from the white paper that shows the classic mistakes with the worst MEIs:

Here is an excerpt that summarizes the average frequency and average severity of the mistakes with the highest MEIs:

Conclusions from the Classic Mistakes Survey

The raw survey results are interesting and so are some of the general trends.

One conclusion is that two of the mistakes added in 2008 (i.e., that weren't in my 1996 book Rapid Development) made the top 10:

• Confusing estimates with targets
• Excessive multi-tasking

This suggests that continued refinement of the classic mistakes list is worthwhile.

A second conclusion is that a few of the mistakes don't occur frequently enough or aren't severe enough when they do occur to really be considered "classic" mistakes:

A third conclusion is that many of the mistakes in the survey do indeed deserve to be called "classic" mistakes. I find it interesting that 8 of the top 10 mistakes in this year's report were listed in a book I published in 1996. If these mistakes were classic in 1996, they're even more classic 12 years later!

Final Thoughts

We'll be updating the classic mistakes survey in 2009, and we'd appreciate your input into the survey. You can take the survey in about 30 minutes. If you take the survey, we'll send you the results before they're made available to the general public.

Why do people keep making these mistakes? I'm interested to hear your thoughts.

Resources

• "Software Development's Classic Mistakes 2008" Construx white paper which includes the complete list of mistakes, descriptions of each mistake, and results from the survey for every classic mistake (login required).
• Classic Mistakes Survey
• My executive presentation on Classic Mistakes (login required)
• My nominations for Top 12 classic mistakes in 1996
• Excerpt from Rapid Development on classic mistakes

My last couple of posts on productivity variations among programmers and the Chief Programmer Team model gave rise to some discussion about hazards of measusring software productivity at the individual programmer level. Software engineering studies normally measure productivity in terms of time to complete a specific task, or sometimes in terms of lines of code per effort-hour, staff-month, or some other measure of effort. Regardless of how you choose to measure productivity, there will be issues.

Productivity in Lines of Code Per Staff Month

Software design is a non-determinisitic activity, and researchers have found 10x variations in the code volume that different designer/developers will generate in response to a particular problem specification. If productivity is measured as lines of code per staff month (or equivalent), that implicitly suggests that the programmer who writes 10 times the amount of code to solve a particular problem is more productive than the programmer who writes 1 times the amount of code. That clearly is not right. Some commenters on my previous blog entry asserted that great programmers always write less code. My observation is that there’s a correlation there, but I wouldn’t make that statement that strongly. I would say that great programmers always write clear code, and that often translates to less code. Sometimes the clearest, simplest, and most obvious design takes a little more code than a design that’s more "clever"--in those cases I think the great programmer will write more code to avoid an overly clever design solution. Regardless, the idea that productivity can be measured cleanly as "lines of code per staff month" is subject to problems either way.

The problem with measuring productivity in terms of lines of code per staff month is the old Dilbert joke about Wally coding himself a minivan. If you measure productivity in terms of volume of code generated, some people will optimize for that measure, i.e., they will find ways to write more lines of code, even if more lines of code aren’t needed. This isn’t really a problem with this specific way of measuring productivity. This really just speaks to the management chestnut that "what gets measured gets done," so you need to be careful what you measure.

Productivity in Function Points

Some of the problems of "lines of code per staff month" can be avoided by measuring program size in function points rather than lines of code. Function points are a "synthetic" measure of program size in which inputs, outputs, queries, and files are counted to determine program size. An inefficient design/coding style won’t generate more function points, so function points aren’t subject to the same issues as lines of code. They are however subject to more practical issues, namely that to get an accurate count of function points you need the services of a certified function point counter (which most organizations don’t have available), and the mapping between how function points are counted and individual work packages is rough enough that it becomes impractical to use them to ascertain the productivity of individual programmers.

What about Complexity?

Managers frequently mention this issue:  "I always give my best programmer the most difficult/most complex sections of code to work on. His productivity on any measured basis might very well be low compared to programmers who get easier assignments, but my other programmers would take twice as long." Yep. That’s a legitimate issue too.

Is There Any Way to Measure Individual Productivity?

Difficulties like these have led many people to conclude that measuring individual productivity is so fraught with problems that no one should even try. I think it is possible to measure individual productivity meaningfully, as long as  you keep several key factors in mind.

1. Don’t expect any single dimensional measure of productivity to give you a very good picture of individual productivity. Think about all the statistics that are collected in sports. We can’t even use a single measure to determine how good a hitter in baseball is. We consider batting average, home runs, runs batted in, on-base percentage, and other factors--and then we still argue about what the numbers mean. If we can’t measure the "good hitter" using a simple measure, why would we expect we could measure something as complex as individual productivity using a simple measure? What we need to do instead is use a combination of measures, which collectively will give us insights into individual productivities. (Measures could include on-time task completion percentage, manager evaluation on a scale of 1-10, peer evaluation on a scale of 1-10, lines of code per staff month, defects reported per line of code, defects, fixed per line of code, bad fix injection rate, etc.)

2. Don’t expect any measures--whether single measures of a combination of measures--to support fine-grain discriminations in productivity among individuals. A good guideline is that measures of individual productivity give you questions to ask but they don’t give you the answers. Using measures of performance for, say, individual performance reviews is both bad management and bad statistics.

3. Remember that trends are usually more important than single-point measures. Measures of individual productivity tend to be far less useful in comparing one individual to another than they are in seeing how one individual is progressing over time.

4. Ask why you need to measure individual productivity at all. In a research setting, researchers need to measure productivity to assess the relative effectiveness of different techniques, and their use of these measures is subject to far fewer problems than measuring individual productivity on real projects is. In a real project environment, what do you want to use the measure(s) for? Performance reviews? Not a good idea for the reasons mentioned above. Task assignments? Most managers I talk with say they *know* who their star contributors are without measuring, and I believe them. Estimation? No, the variations caused by different design approaches, different task difficulty, and related factors make that an ineffective way to build up project estimates.

On real projects it’s hard to find a use for individual productivity measures that is both useful and statistically valid. In my experience, aside from research settings the attempt to measure individual performance arises most often from a desire to do something with the measurements that isn’t statistically valid. So while I see the value of measuring individual performance in research settings, I think it’s difficult to find cases in which the effort is justified on real projects.

(Measuring team productivity and organizational productivity is a different matter -- I'll blog about that soon).

One spinoff from the 10x difference in programmer productivity was the Chief Programmer Team structure. The idea of the chief-programmer team was originally developed at IBM during the late 1960s (Baker 1972, Baker and Mills 1973). It was popularized by Fred Brooks in the Mythical Man-Month (Brooks 1975, 1995), in which Brooks referred to it as a surgical team. The two terms are interchangeable. I described the technique in my 1996 book Rapid Development, but I think we've learned some important lessons about the CPT structure since then.

Original Chief Programmer Team Project

The original chief programmer team project was conducted in the late 1960s. IBM commissioned to build an information retrieval system for the New York Times. The Chief Programmer on that project (the original Chief Programmer) was Harlan Mills, who created all the design and wrote all of the production code. He had eight other people arrayed around him in various support functions:

• A "backup programmer" serves as the chief programmer’s alter ego. The backup programmer supports the chief programmer as critic, research assistant, technical contact for outside groups, and backup-up chief.
• The "administrator" handles administrative matters such as money, people, space, and machines. The Chief has ultimate say about these matters, but the administrator frees the Chief from having to deal with them on a daily basis.
• The "toolsmith" is responsible for creating custom tools requested by the Chief . In today’s terminology, the toolsmith would be in charge of maintaining the build environment, creating scripts, etc.  
• The team is rounded out by a "language lawyer" who supports the Chief by answering esoteric questions about the programming language the Chief is using.

Several of the support roles suggested in the original chief-programmer proposal are now regularly performed by nonprogrammers--by documentation specialists, test specialists, and program managers. Other tasks such as word processing and version control have been simplified so much by modern software tools that they no longer need to be performed by support personnel. And the internet has reduced the need for language lawyers--many questions can be answered via a quick search on the web.

Attempts to Replicate IBM’s Chief Programmer Team Results: Is 10x Good Enough?

On the original project, Harlan Mills personally wrote 83,000 lines of production code in one year. He wrote that code on a batch mode operating system. And on punch cards! Even when you divide the 83,000 lines of code by the nine people on the project, that’s 9,200 lines of code per staff year, which is still in the ballpark of acceptable productivity for similar kinds of projects 40 years later. With productivity like that under those circumstances you can see why the IBM project was heralded as one of the most successful projects of its time.

In the years since that project many organizations have tried to implement Chief Programmer teams, and few have been able to repeat IBM’s initial stunning success. The achilles heel of the Chief Programmer Team model is that for it to make sense to organize staff the way they were organized on the IBM project, the Chief Programmer has to be more productive than everyone else on the team put together. On the original IBM project, Harlan Mills was a near-genius programmer who was an expert software methodologist, talented writer, exceptionally self-disciplined, and highly motivated. When he decided to roll up his sleeves and write code, he had few peers. Think "Kent Beck of His Day" and you’d be pretty close. He was one of the rare individuals truly capable of doing more work than the eight other people on his team put together.

In a previous blog posting I discussed the fact that numerous studies have found 10-fold variations in productivity between the best and worst programmers with similar levels of experience. For the CPT model to work, the Chief Programmer doesn’t have to be 10x as productive as the worst programmer. He has to do the work of eight or nine people put together, which means he has to be 10x as productive as the average programmer, not 10x as productive as the worst. That’s a very tall order. If you assume the best programmer is 10x as productive as the worst, then the best will be only something like 2-3 as productive as the average programmer, and that isn’t good enough for the CPT model to pay off with a total project team of nine people.

Another factor is that, while numerous studies have found 10x differences among individuals, researchers have not found 10-fold differences among programmers working within the same organizations. Some research has found that good programmers tend to cluster within certain companies, average programmers tend to cluster within other companies, and so on (Mills 1983). So even if there’s a 10x difference industrywide, the difference you’d typically see within a given company is more like 3-5x from best to worst, which means the difference from best to average is more like 1.5x or 2x within any given company.

Bottom line: The Chief Programmer Team organization can make sense in the rare case in which you have a near genius on your staff--one that is dramatically more productive than the average programmer on your staff. But from I've seen there are far fewer near geniuses than there are near genius wannabees, and that limits the applicability of the technique.

Resources 

• Construx's 10x Software Development Seminar. My company's answer to the question of "What does it take to move a team toward the 10x end of the scale?"
• My earlier earlier blog posting on 10x differences in productivity.
• My earlier blog posting on self-education.
• Construx's web resources for professional development.

References

Brooks, Frederick P., Jr. The Mythical Man-Month, Reading Massachusetts: Addison-Wesley, 1975.

Brooks, Frederick P., Jr. The Mythical Man-Month, Anniversary Edition, Reading Massachusetts: Addison-Wesley, 1995.

Baker, F. Terry. "Chief Programmer Team Management of Production Programming," IBM Systems Journal, vol. 11, no. 1, 1972, pp. 56-73.

Baker, F. Terry and Harlan D. Mills. "Chief Programmer Teams." Datamation, Volume 19, Number 12 (December 1973), pp. 58-61.

McConnell, Steve. Rapid Development. Microsoft Press, 1996.

Mills, Harlan D. Software Productivity, Boston, Massachusetts: Little, Brown, 1983.

Some blog readers have asked for more background on where the "10x" name of this blog cam from. The gist of the name is that researchers have found 10-fold differences in productivity and quality between different programmers with the same levels of experience and also between different teams working within the same industries.

Individual Productivity Variation in Software Development

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968). They studied professional programmers with an average of 7 years’ experience and found that the ratio of initial coding time between the best and worst programmers was about 20 to 1; the ratio of debugging times over 25 to 1; of program size 5 to 1; and of program execution speed about 10 to 1. They found no relationship between a programmer’s amount of experience and code quality or productivity.

Detailed examination of Sackman, Erickson, and Grant's findings shows some  flaws in their methodology (including combining results from programmers working in low level programming languages with those working in high level programming languages). However, even after accounting for the flaws, their data still shows more than a 10-fold difference between the best programmers and the worst.

In years since the original study, the general finding that "There are order-of-magnitude differences among programmers" has been confirmed by many other studies of professional programmers (Curtis 1981, Mills 1983, DeMarco and Lister 1985, Curtis et al. 1986, Card 1987, Boehm and Papaccio 1988, Valett and McGarry 1989, Boehm et al 2000).

There is also lots of anecdotal support for the large variation between programmers. During the time I was at Boeing in the mid 1980s, there was a project that had about 80 programmers working on it that was at risk of missing a critical deadline. The project was critical to Boeing, and so they moved most of the 80 people off that project and brought in one guy who finished all the coding and delivered the software on time. I didn't work on that project, and I didn't know the guy, so I'm not 100% sure the story is even true. But I heard the story from someone I trusted, and it seemed true at the time.

This degree of variation isn't unique to software. A study by Norm Augustine found that in a variety of professions--writing, football, invention, police work, and other occupations--the top 20 percent of the people produced about 50 percent of the output, whether the output is touchdowns, patents, solved cases, or software (Augustine 1979). When you think about it, this just makes sense. We've all known people who are exceptional students, exceptional athletes, exceptional artists, exceptional parents--these differences are just part of the human experience; why would we expect software development to be any different?

Extremes in Individual Variation on the Bad Side

Augustine's study observed that, since some people make no tangible contribution whatsoever (quarterbacks who make no touchdowns, inventors who own no patents, detectives who don’t close cases, and so on), the data probably understates the actual variation in productivity.

This appears to be true in software. In several of the published studies on software productivity, about 10% of the subjects in the experiments weren't able to complete the experimental assignment. In the studies, they writeups say, "Therefore those experimental subjects' results were excluded from our data set." But in real life if someone "doesn't complete the assignment" you can't just "exclude their results from the data set." You have to wait for them to finish, assign someone else to do their work, and so on. The interesting (and frightening) implication of this is that something like 10% of the people working in the software field might actually be contributing *negative& productivity to their projects. Again, this lines up well with real-world experience. I think many of us can think of specific people we've wworked with who fit that description.

Team Productivity Variation in Software Development

Software experts have long observed that team productivity varies about as much as individual productivity does--by an order of magnitude (Mills 1983). Part of the reason is that good programmers tend to cluster in some organizations, and bad programmers tend to cluster in other organizations, an observation that has been confirmed by a study of 166 professional programmers from 18 organizations (Demarco and Lister 1999).

In one study of seven identical projects, the efforts expended varied by a factor of 3.4 to 1 and program sizes by a factor of 3 to 1 (Boehm, Gray, and Seewaldt 1984). In spite of the productivity range, the programmers in this study were not a diverse group. They were all professional programmers with several years of experience who were enrolled in a computer-science graduate program. It’s reasonable to assume that a study of a less homogeneous group would turn up even greater differences.
 
An earlier study of programming teams observed a 5-to-1 difference in program size and a 2.6-to-1 variation in the time required for a team to complete the same project (Weinberg and Schulman 1974).

After reviewing data more than 20 years of data in constructing the Cocomo II estimation model, Barry Boehm and other researchers concluded that developing a program with a team in the 15th percentile of programmers ranked by ability typically requires about 3.5 times as many staff-months as developing a program with a team in the 90th percentile (Boehm et al 2000). The difference will be much greater if one team is more experienced than the other in the programming language or in the application area or in both.

One specific data point is the difference in productivity between Lotus 123 version 3 and Microsoft Excel 3.0. Both were desktop spreadsheet applications completed in the 1989-1990 timeframe. Finding cases in which two companies publish data on such similar projects is rare, which makes this head-to-head comparison especially interesting. The results of these two projects were as follows: Excel took 50 staff years to produce 649,000 lines of code. Lotus 123 took 260 staff years to produce 400,000 lines of code. Excel's team produced about 13,000 lines of code per staff year. Lotus's team produced 1,500 lines of code per staff year. The difference in productivity between the two teams was more than a factor of 8, which supports the general claim of order-of-magnitude differences not just between different individuals but also between different project teams.

What Have You Seen?

Have you seen 10;1 differences in capabilities between different individuals? Between different teams? How much better was the best programmer you've worked with than the worst? Does 10:1 even cover the range?

I look forward to hearing your thoughts.

References

Augustine, N. R. 1979. "Augustine’s Laws and Major System Development Programs." Defense Systems Management Review: 50-76.

Boehm, Barry W., and Philip N. Papaccio. 1988. "Understanding and Controlling Software Costs." IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Boehm, Barry W., T. E. Gray, and T. Seewaldt. 1984. "Prototyping Versus Specifying: A Multiproject Experiment." IEEE Transactions on Software Engineering SE-10, no. 3 (May): 290-303. Also in Jones 1986b.

Card, David N. 1987. "A Software Technology Evaluation Program." Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. "Substantiating Programmer Variability." Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. "Software Psychology: The Need for an Interdisciplinary Program." Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. "Programmer Performance and the Effects of the Workplace." Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

DeMarco, Tom and Timothy Lister, 1999. Peopleware: Productive Projects and Teams, 2d Ed. New York: Dorset House, 1999.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Sackman, H., W.J. Erikson, and E. E. Grant. 1968. "Exploratory Experimental Studies Comparing Online and Offline Programming Performance." Communications of the ACM 11, no. 1 (January): 3-11.

Valett, J., and F. E. McGarry. 1989. "A Summary of Software Measurement Experiences in the Software Engineering Laboratory." Journal of Systems and Software 9, no. 2 (February): 137-48.

Weinberg, Gerald M., and Edward L. Schulman. 1974. "Goals and Performance in Computer Programming." Human Factors 16, no. 1 (February): 70-77.

The question of how to scale up quickly in a software startup company is a perennially tough issue. There are some good ways to get started -- starting with a core of really senior people is one time-honored approach. Starting with a core team of people who have worked together at another employer is another approach that often works. The question, though, is how do you scale up beyond that core, and how do you scale up quickly?

I think it's an especially tough issue for people who are process oriented. If an organization is already pretty good sized, then having well defined and efficient processes can be supportive of scaling up quickly. Telecordia added something like 1000 people to its technical staff in the year it was first assessed at CMM Level 5. But that's a very large organization to start with. If you're in startup mode I think it's hard to add staff quickly without your organization's software practices reverting to whatever the industry mean in your geographic area is -- the new staff just has too strong a dilution effect on the existing staff for it to work any other way.

Trying to startup quickly by outsourcing is a dead end as far as I'm concerned, especially to India where turnover is so high. Offshore captives can work, but minimum workable size seems to be about 100 people, and it probably takes 2-3 years of ramp up to get to financial break even. It's hard to find a time-to-market gain in this approach.

I'll be in New York City next week teaching "Software Estimation in Depth." This is an enjoyable class to teach. It has great lab exercises, and it's fun to see the lightbulbs going off in people's heads as they "get" the key concepts in software estimation. You can read more about the class here: http://www.construx.com/Page.aspx?nid=17&id=32.

My company's also teaching several other classes in New York City next week (the only time in 2008 we'll be doing open-enrollment seminars on the east coast). Other classes are:

• How to be Agile without Being Extreme
• 10x Software Engineering
• Requirements Boot Camp
• Software Project Management Boot Camp
• Design Boot Camp

You can read more about all these classes at http://www.construx.com/Page.aspx?nid=387.

Here are more detailed summaries of these classes.

Software Estimation in Depth, March 24-25, 2008, Steve McConnell, Instructor  Details

This course focuses on providing many useful rules of thumb and procedures for creating software estimates ("the art of estimation") and a brief introduction to mathematical approaches to creating software project estimates ("the science of estimation"). This course provides techniques for making sure estimation is treated as an analytical rather than a political process. It explains how to negotiate effectively with other project stakeholders (such as marketing, management and your clients) so that everyone wins. The course features extensive lab work to give you hands-on experience creating many different kinds of software estimates. This seminar will be taught by Steve McConnell, author of Code Complete, Rapid Development, and Software Estimation: Demystifying the Black Art. More >

How to be Agile Without Being Extreme, March 24-25, 2008, Jerry Deville, Instructor  Details

Agile software development promises low overhead, high flexibility, and satisfied customers, but how do you separate the hype from the reality? Leading organizations have benefited from Agile development practices for many years. Learn how to select and deploy today’s most powerful Agile practices. Apply the essentials of Scrum, Extreme Programming, Crystal, Lean, and other Agile methods. This intensive seminar presents modern practices combined with decades of time-tested, low-risk methods–all with a track record of proven results. This seminar is based on Construx’s experience working with companies that have successfully deployed Agile practices--and our experiences with companies whose agile projects have failed. Extensive case studies and hands-on exercises will show you how to select and apply the particular Agile development techniques that are best for your specific projects. More >

10x Software Engineering, March 26-28, 2008, Matt Peloquin, Instructor  Details

Decades of research have found at least a ten-fold “10x” difference in productivity and quality between the best developers and the worst–and between the best teams and the worst. Discover the 5 Key Principles of 10x Engineering and avoid the productivity traps of “minus-x” engineering. Practice critical techniques that will turn your team into a high performing, 10x Team. More >

Requirements Boot Camp, March 26-28, 2008, Earl Beede, Instructor  Details

What is the most frequently reported cause of software project failure–regardless of project size or type of software? Requirements challenges. Discover how leading-edge companies use requirements engineering to support successful software projects. Learn the three purposes of requirements and how to distinguish between requirements fantasies and requirements reality. Practice practical techniques for exploring user needs, capturing requirements, controlling changes, and building highly satisfactory software. More >

Software Project Management Boot Camp, March 26-28, 2008, Jerry Deville, Instructor  Details

Leading any project can be a challenge. Leading a software project can be even more challenging if you're new to project management or new to software. This seminar will help you make the transition to solid software project leadership. Software Project Management Boot Camp teaches you the concepts and techniques necessary to manage projects successfully. This seminar closely follows the Project Management Institute's (PMI) Project Management Body of Knowledge (PM-BOK) and shows how to apply these best practices to a typical small to medium sized software project. More >

Design Boot Camp, March 26-28, 2008, Steve Tockey, Instructor Details

Different designers will create designs that differ by at least a factor of 10 in the code volume produced. How do you invent simple, straightforward designs and avoid complex, error-prone designs? Understand the fundamental design principles that lead to high-quality designs requiring low implementation effort. Learn both Agile and traditional approaches to create great designs quickly and economically. More >

[This is an expansion of one of my comments on an earlier Technical Debt post]

This leads us to the third issue that should be considered: Is there a path that is quicker than the good path and that won't affect the rest of the system? In other words, is there a quick path that can be isolated from the rest of the system in such a way that it doesn't create any ongoing interest payment/make other work more difficult? In my experience teams often turn the technical debt decision into a simplistic "two option" decision -- good path vs. quick and dirty path. Pushing through to a third option is important because often the best path is the one that is fairly quick, albeit not as quick as the pure quick and dirty path, and whose adverse affects are better contained than those of the pure quick and dirty path.

In this example, either Option 2 or Option 3 is an attractive short-term alternative to Option 1. That is, either $1200 or $1800 is a fraction of the cost/effort of $6000. But if you select Option 2 you saddle yourself with an obligation to revise the code later--either you reimplement the good solution, which costs more, or you keep paying interest, which costs more. When you select Option 3 you introduce the possibility of choosing never to pay off the technical debt, because there isn't any ongoing penalty, and so there isn't any urgency to pay off the debt.

Bottom line: When facing the prospect of taking on technical debt, be sure to generate more than two design options. Don't oversimplify technical debt decision making to just the two extremes.

The term "technical debt" was coined by Ward Cunningham to describe the obligation that a software organization incurs when it chooses a design or construction approach that's expedient in the short term but that increases complexity and is more costly in the long term.

Ward didn't develop the metaphor in very much depth. The few other people who have discussed technical debt seem to use the metaphor mainly to communicate the concept to technical staff. I agree that it's a useful metaphor for communicating with technical staff, but I'm more interested in the metaphor's incredibly rich ability to explain a critical technical concept to non-technical project stakeholders.

What is Technical Debt? Two Basic Kinds

The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job. In some cases, this kind of debt can be incurred unknowingly, for example, your company might acquire a company that has accumulated significant technical debt that you don't identify until after the acquisition. Sometimes, ironically, this debt can be created when a team stumbles in its efforts to rewrite a debt-laden platform and inadvertently creates more debt. We'll call this general category of debt Type I.

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. "If we don't get this release done on time, there won't be a next release" is a common refrain—and often a compelling one. This leads to decisions like, "We don't have time to reconcile these two databases, so we'll write some glue code that keeps them synchronized for now and reconcile them after we ship." Or "We have some code written by a contractor that doesn't follow our coding standards; we'll clean that up later." Or "We didn't have time to write all the unit tests for the code we wrote the last 2 months of the project. We'll right those tests after the release." (We'll call this Type II.)

The rest of my comments focus on the kind of technical debt that's incurred for strategic reasons (Type II).

Short-Term vs. Long-Term Debt

With real debt, a company will maintain both short-term and long-term debt. You use short-term debt to cover things like gaps between your receivables (payments from customers) and expenses (payroll). You take on short term debt when you have the money, you just don't have it now. Short-term debt is expected to be paid off frequently. The technical equivalent seems straightforward. Short-term debt is the debt that's taken on tactically and reactively, usually as a late-stage measure to get a specific release out the door. (We'll call this Type II.A.)

Long term debt is the debt a company takes on strategically and proactively--investing in new capital equipment, like a new factory, or a new corporate campus. Again, the technical equivalent seems straightforward: "We don't think we're going to need to support a second platform for at least five years, so this release can be built on the assumption that we're supporting only one platform." (We'll call this Type II.B.)

The implication is that short-term debt should be paid off quickly, perhaps as the first part of the next release cycle, whereas long-term debt can be carried for a few years or longer.

Incurring Technical Debt

When technical debt is incurred for strategic reasons, the fundamental reason is always that the cost of development work today is seen as more expensive than the cost will be in the future. This can be true for any of several reasons.

Time to Market. When time to market is critical, incurring an extra $1 in development might equate to a loss of $10 in revenue. Even if the development cost for the same work rises to $5 later, incurring the $1 debt now is a good business decision.

Preservation of Startup Capital. In a startup environment you have a fixed amount of seed money, and every dollar counts. If you can delay an expense for a year or two you can pay for that expense out of a greater amount of money later rather than out of precious startup funds now.

Delaying Development Expense. When a system is retired, all of the system's technical debt is retired with it. Once a system has been taken out of production, there's no difference between a "clean and correct" solution and a "quick and dirty" solution. Unlike financial debt, when a system is retired all its technical debt is retired with it. Consequently near the end of a system's service life it becomes increasingly difficult to cost-justify investing in anything other than what's most expedient.

Be Sure You Are Incurring The Right Kind of Technical Debt

Some debt is taken on in large chunks: "We don't have time to implement this the right way; just hack it in and we'll fix it after we ship." Conceptually this is like buying a car—it's a large debt that can be tracked and managed. (We'll call this Type II.A.1.)

Other debt accumulates from taking hundreds or thousands of small shortcuts--generic variable names, sparse comments, creating one class in a case where you should create two, not following coding conventions, and so on. This kind of debt is like credit card debt. It's easy to incur unintentionally, it adds up faster than you think, and it's harder to track and manage after it has been incurred. (We'll call this Type II.A.2.)

Both of these kinds of debt are commonly incurred in response to the directive to "Get it out the door as quickly as possible." However, the second kind (II.A.2) doesn't pay off even in the short term of an initial development cycle and should be avoided.

Debt Service

One of the important implications of technical debt is that it must be serviced, i.e., once you incur a debt there will be interest charges.

If the debt grows large enough, eventually the company will spend more on servicing its debt than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (i.e., "servicing the debt") that there is little time left over to add new capabilities to the system. With financial debt, analysts talk about the "debt ratio," which is equal to total debt divided by total assets. Higher debt ratios are seen as more risky, which seems true for technical debt, too.

Attitudes Toward Technical Debt

Like financial debt, different companies have different philosophies about the usefulness of debt. Some companies want to avoid taking on any debt at all; others see debt as a useful tool and just want to know how to use debt wisely.

I've found that business staff generally seems to have a higher tolerance for technical debt than technical staff does. Business executives tend to want to understand the tradeoffs involved, whereas some technical staff seem to believe that the only correct amount of technical debt is zero.

The reason most often cited by technical staff for avoiding debt altogether is the challenge of communicating the existence of technical debt to business staff and the challenge of helping business staff remember the implications of the technical debt that has previously been incurred. Everyone agrees that it's a good idea to incur debt late in a release cycle, but business staff can sometimes resist accounting for the time needed to pay off the debt on the next release cycle. The main issue seems to be that, unlike financial debt, technical debt is much less visible, and so people have an easier time ignoring it.

How do You Make an Organization's Debt Load More Visible?

One organization we've worked with maintains a debt list within its defect tracking system. Each time a debt is incurred, the tasks needed to pay off that debt are entered into the system along with an estimated effort and schedule. The debt backlog is then tracked, and any unresolved debt more than 90 days old is treated as critical.

Another organization maintains its debt list as part of its Scrum product backlog, with similar estimates of effort required to pay off each debt.

Either of these approaches can be used to increase visibility into the debt load and into the debt service work that needs to occur within or across release cycles. Each also provides a useful safeguard against accumulating the "credit card debt" of a mountain of tiny shortcuts mentioned earlier. You can simply tell the team, "If the shortcut you are considering taking is too minor to add to the debt-service defect list/product backlog, then it's too minor to make a difference; don't take that shortcut. We only want to take shortcuts that we can track and repair later."

Ability to Take on Debt Safely Varies

Different teams will have different technical debt credit ratings. The credit rating reflects a team's ability to pay off technical debt after it has been incurred.

A key factor in ability to pay off technical debt is the level of debt a team takes on unintentionally, i.e., how much of its debt is Type I? The less debt a team creates for itself through unintentional low-quality work, the more debt a team can safely absorb for strategic reasons. This is true regardless of whether we're talking about taking on Type I vs. Type II debt or whether we're talking about taking on Type II.A.1 vs. Type II.A.2 debt.

One company tracks debt vs. team velocity. Once a team's velocity begins to drop as a result of servicing its technical debt, the team focuses on reducing its debt until its velocity recovers. Another approach is to track rework, and use that as a measure of how much debt a team is accumulating.

Retiring Debt

"Working off debt" can be motivational and good for team morale. A good approach when short-term debt has been incurred is to take the first development iteration after a release and devote that to paying off short-term technical debt.

The ability to pay off debt depends at least in part on the kind of software the team is working on. If a team incurs short-term debt on a web application, a new release can easily be rolled up after the team backfills its debt-reduction work. If a team incurs short-term debt in avionics firmware— the pay off of which which requires replacing a box on an airplane— that team should have a higher bar for taking on any short-term debt. This is like a minimum payment--if your minimum payment is 3% of your balance, that's no problem. If the minimum payment is $1000 regardless of your balance, you'd think hard about taking on any debt at all.

Communicating about Technical Debt

The technical debt vocabulary provides a way to communicate with non-technical staff in an area that has traditionally suffered from a lack of transparency. Shifting the dialog from a technical vocabulary to a financial vocabulary provides a clearer, more understandable framework for these discussions. Although the technical debt terminology is not currently in widespread use, I've found that it resonates immediately with every executive I've presented it to as well as other non-technical stakeholders. It also makes sense to technical staff who are often all-too-aware of the debt load their organization is carrying.

Here are some suggestions for communicating about debt with non-technical stakeholders:

Use an organization's maintenance budget as a rough proxy for it's technical debt service load. However you will need to differentiate between maintenance that keeps a production system running vs. maintenance that extends the capabilities of a production system. Only the first category counts as technical debt.

Discuss debt in terms of money rather than in terms of features. For example, "40% of our current R&D budget is going into supporting previous releases" or "We're currently spending $2.3 million per year servicing our technical debt."

Be sure you're taking on the right kind of debt. Not all debts are equal. Some debts are the result of good business decisions; others are the result of sloppy technical practices or bad communication about what debt the business intends to take on. The only kinds that are really healthy are Types II.A.1 and II.B.

Treat the discussion about debt as an ongoing dialog rather than a single discussion. You might need several discussions before the nuances of the metaphor fully sink in.

Technical Debt Taxonomy

Here's a summary of the kinds of technical debt:

Non Debt

   Feature backlog, deferred features, cut features, etc. Not all incomplete work is debt. These aren't debt, because they don't require interest payments.

Debt

   I. Debt incurred unintentionally due to low quality work

   II. Debt incurred intentionally

      II.A. Short-term debt, usually incurred reactively, for tactical reasons

         II.A.1. Individually identifiable shortcuts (like a car loan)

         II.A.2. Numerous tiny shortcuts (like credit card debt)

      II.B. Long-term debt, usually incurred proactively, for strategic reasons

Summary

What do you think? Do you like the technical debt metaphor? Do you think it's a useful way to communicate the implications of technical/business decision making to non-technical project stakeholders? What's your experience? I look forward to your thoughts.

Resources

• Ward Cunningham's OOPSLA '92 Experience Report that first mentions technical debt.

• Martin Fowler's brief bliki entry about technical debt.

• c2 wiki discussions of Complexity As Debt and Technical Debt.

PM*Boulevard interviewed me earlier this summer about Agile development. Below I've excerpted the PM*Boulevard interview, updated some of my answers, and added a little additional commentary.

5Qs on Agile with Steve McConnell
  
Readers of Software Development magazine once named Steve McConnell one of the three most influential people in the software industry. The CEO and Chief Software Engineer at Construx Software, Steve has generously agreed to kick off our "5Qs on Agile" feature by answering the following five often-asked questions about Agile development.

Q1: Why use Agile methods?
Agile methods focus on shorter iterations, in which the software is brought to a releasable level of quality fairly often, usually somewhere between weekly and monthly. Short iterations provide numerous technical and management benefits. On the technical side, the main benefit is reduced integration risk because the amount of software being integrated is kept small. Short iterations also help to keep quality under control by driving to a releasable state frequently, which prevents a project from accumulating a large backlog of defect correction work. On the management side, the frequent iterations provide frequent evidence of progress, which tends to lead to good status visibility, good customer relations, and good team morale.

Agile methods also usually treat requirements as more dynamic than traditional methods do. For some environments that's a plus and for some it's a minus. If you're working in an environment that doesn't need a lot of long range predictability in its feature set, treating requirements dynamically can save a lot of detailed requirements specification work and avoid the "requirements spoilage" that often goes along with working through a lengthy backlog of detailed requirements.

Q2: What is the biggest challenge when implementing Agile methods?
The biggest challenge we see in our consulting and training business is walking the walk. You can't just say you're doing Agile. You have to follow through with specific actions. Of course that's the same problem we saw years ago with object oriented methods, and before that with structured methods, so that problem isn't unique to Agile.

The most common specific challenges we see are simply the challenges of "turning the battleship" in a large organization to overcome the inertia of entrenched work practices and expectations and getting reoriented to do things in a different way. You have to muster the resolve to actually work in short iterations. You have to build frequently, at least every day, and you have to develop the discipline to keep the build healthy. You have to push each iteration to a releasable level of quality even if that's hard to do at first. As before, this problem isn't unique to Agile. If we're working with an organization and find that their biggest need is to do a better job of defining requirements up front (which isn't very agile), "turning the battleship" to define better requirements up front will be just as hard.

Q3: In what environments will Agile be most successful?
Full-blown Agile seems to me to be best suited for environments in which the budget is fixed on an annual basis, team sizes are fixed on an annual basis (because of the budget), and the project staff's mission is to deliver the most valuable business functionality that they can deliver in a year's time with a fixed team size. This mostly describes in-house, business systems dev organizations.

Full-blown agile (especially the flexible requirements part) is less-well suited to companies that sell software, because maintaining a lot of requirements flexibility runs counter to the goal of providing mid-term and long-term predictability. We've found that many organizations value predictability more than they value requirements flexibility. That is, they value the ability to make commitments to key customers or to the marketplace more than they value the ability to "respond to change."

For anything less than full-blown Agile, however, we find that many agile practices are well-suited to the vast majority of environments. Short development iterations are nearly always valuable, regardless of whether you define 5% of your requirements up front or 95% up front. Keeping the software close to a releasable level of quality at all times is virtually always valuable. Scrum as a management style and discipline seems to be very broadly applicable. Small, empowered teams are nearly always useful. I go into more detail on the detailed strengths and weaknesses of specific agile development practices in my executive presentation on The Legacy of Agile Development.

Q4: What is the future of Agile?
Agile has largely become a synonym for "modern software practices that work," so I think the future of Agile with a capital "A" is the same as the past of Object Oriented or Structured. We rarely talk about Object Oriented programming anymore; it's just programming. Similarly, I think Agile has worked its way into the mainstream such that within the next few years we won't talk about Agile much anymore; we'll just talk about programming, and it will be assumed that everyone means Agile whenever that's appropriate.

Q5: Can you recommend a book, blog, podcast, Web site, or other information source to our readers that you find interesting or intriguing right now?
I'm most excited about the Software Development Best Practices discussion forum that we launched a few weeks ago. That's at http://forums.construx.com/ . I also started blogging recently, and you can read my blog at http://blogs.construx.com/blogs/stevemcc/default.aspx . Feel free to contact me by e-mail at stevemcc@construx.com. 

Additional Commentary (October 2007)
After this interview posted back in August 2007 I received an interesting email that said, "Wow, you seem really pro-Agile now. What happened?"

I was surprised at that email because I didn't think my comments in the 5Qs were especially "pro agile." I thought they emphasized the strengths of agile and also some of the common failure modes. Another reason that comment was interesting was the hint that I'd been "anti-agile" before. I've never been either pro-agile or anti-agile -- I've always been pro-whatever-practices-work-best. In many situations the practices that work best are the practices that today are associated with agile development. And in some circumstances, other older practices still work best.

So I'm not pro-agile or anti-agile. I'm not pro-CMM or anti-CMM, pro-BDUF or anti-BDUF, or pro-pair programming or anti-pair programming. For that matter I'm not even pro-waterfall or anti-waterfall. At Construx we've worked with such a tremendous variety of companies over the years that we've encountered at least one environment in which each of these practices will work best. The big wide world of software projects is amazingly diverse, and that calls for software development practices that are just as amazingly diverse. Agile has simply added more tools to the toolbox so that we have a richer set from which to choose the right tool for any particular job.

Resources

• Legacy of Agile Development Executive presentation
• PM*Boulevard's original 5Qs about agile interview with me. (Did the original interview really seem rabidly pro agile?)
• Other PM*Boulevard 5Q interviews about Agile
• Construx's How to Be Agile Without Being Extreme seminar
• Construx's Agile Practices Review/Agile Adoption

Also Known as: How I Spent My Summer Vacation

My big project this summer was building a fort for my kids. I'd wanted to build a clubhouse or treehouse or fort or something for the past few years, but we didn't have a good place to put it. Then while clearing some blackberries in the spring I discovered that our property extended about 20 feet further into the adjacent overgrown area than I had thought, and that was the perfect place for a fort.

Whenever I do a physical construction project like this I try to pay attention to which attributes of the project are similar to software projects and which are different. The comparisons are made more challenging by the fact that my construction projects are recreational, whereas I'm trying to draw comparisons to commercial software projects. For the first half of the project, no good similarities jumped at out me. But as the project started to take much longer than I expected, I began to see more and more similarities between my estimates on the fort and problems people run into with software estimates.

Original Work Plan

Here was the work plan I had carried around in my head for a few weeks before I started the project:

Day 1: Dig holes for footings, pour concrete for footings, haul building materials from my driveway up the hill to the fort.
Day 2: Cut posts and beams to length. This was planned as a half day because I didn't want to put too much stress on the concrete footings until Day 3.
Day 3: Finish beams and joists and install decking; do some of the deck railings as time permits
Day 4: Complete the fort's framing, minus the roof
Day 5: Frame and install the roof
Day 6: Install door and windows; finish deck railing; install trim boards
Part time the next couple of weeks: Finish loose ends

I won't go through all the errors in my estimates for the whole project, but let's take a look at what I really did on Day 1.

DAY 1

Task 1.1 Clear brush from site (~1 hour). I'd known that I had a little brush still to clear, but I thought it would take me about 10-15 minutes. Once I started looking at where I needed to put the footings, I found that I really couldn't put them where I'd planned because I would be inside the setback for the property. So I needed to move the fort back about 5 feet, and that meant clearing a bunch of brush including scrubby trees that I hadn't planned to clear.

1.2 Survey the site and determine placement of footings (~3 hours). I'd originally planned to build the deck with 2 beams and 2 posts per beam. After looking at some span tables, I concluded that I could *probably* get away with 2 beams with 2 posts each, but what I was building was right on the border between 2 and 3 beams and between 2 and 3 posts per beam. I decided to err on the side of caution, and that meant I needed 9 footings instead of 4. Meanwhile, I had never really adjusted my time expectations to digging 9 holes instead of 4. Siting the 9 holes also turned out to be an issue because of a big stump in the middle of my area.

The site overall had more of a slope than I had realized. I wanted to stake out the corners and use string to locate the position of each hole and make sure the holes were square. Due to the slope, the stakes I was using weren't tall enough on the downhill side of the site, and I spent time pounding in stakes that ended up not being tall enough, then pulling them out, hammering together makeshift taller stakes, and then pounding those in.

I ended up spending a lot of time moving stakes and string around trying to figure out how to get 9 holes that were (a) not blocked by roots from the stump, (b) not blocked by roots from the tree in back of the fort, (c) far enough back from the property line to meet the setback requirement, (d) square relative to each other (which was hard to determine at this stage because of the slope I was building on).

1.3 Dig post holes (~2 hours). I had to dig 9 holes, 12" in diameter, 24" deep. This actually went quicker than I expected. I used a clamshell digger and for the holes where I didn't run into any roots it was something like 5 minutes per hole. The difficult holes were the holes where I ran into roots partway down and then had to hack them out. Some of the holes had quite a few roots.

1.4 Haul 20 80# bags of concrete up the hill (~1.5 hours). I had originally thought I could haul the concrete up the hill using a wheelbarrow, but the hill was just too steep. So I had to hand carry each 80# bag one at a time. It was also about 80 degrees and 95% relative humidity at this point, which meant I needed to rest and drink water every couple of bags. The change from 4 holes to 9 holes also increased the number of bags I had to haul from about 10 to about 20.

1.5 Pour 2 Footings (1.5 hours). At this point I was pretty worn out, but I also really wanted the feeling of completion from pouring at least one of the footings. So I ended up pouring 2 of the footings and calling it a day since there was no way I was going to complete all 9 of them at that point in the day.

End of Day 1. The picture below shows how far I got at the end of Day 1.

What Went Wrong with My Estimate for Day 1

• I hadn't examined my planned site well enough to know what I didn't know -- i.e., my originally planned site wouldn't work and I didn't understand how much slope there was.
• I never revised the expectations I had created while planning a 4-footing Day 1 to more appropriate expectations for a 9-footing Day 1. That one mistake affected my site layout, concrete hauling, hole digging, and concrete pouring.
• Brush clearing just took longer than I expected, and I hadn't included it in my estimate at all.
• Surveying the site also just took longer than I expected, and would have even without the change from 4 holes to 9.

DAY 2

What I could complete on Day 2 was limited by the fact that hadn't poured all the footings on Day 1, so about all I could do on Day 2 was pour the remaining 7 footings and haul the rest of the building materials up the hill. The rest of the footing pouring went fine and took about 4 hours. Then I needed to haul the materials up from the driveway. The pile of stuff didn't look all that intimidating:

Superficial appearances aside, however, there are 10 16' 2x8 pressure treated joists in that pile, and those suckers are heavy. There are also 3 12' 4x8 pressure treated beams in that pile, and those suckers are *really* heavy! And then there were 70 2x4s and 50 lengths of 5/4" decking, and 100 2x2 balusters for the railing, and about 15 sheets of plywood, and 2 bundles of roofing shingles, and a lot of other stuff, and all this stuff just starts to add up after awhile. It took me at least 50 trips up the hill, and that ended up taking me about 3 hours.

At the end of Day 2 I was about where I thought I'd be at the end of Day 1 after 2 pretty long days. For the record, here's what I had done at the end of Day 2:

What Went Wrong with the rest of My Estimate for Day 1 (i.e., the Work I Did on Day 2)

• Hauling the building materials up the hill took longer than I planned, mostly because I'd never bothered to break down the "hauling" task and realize that it was going to take 50 trips, not 10.

DAYS 3-6

Days 3-6 went about like Days 1 & 2 had gone, which is to say there were lots of little tasks that turned out to be medium-sized tasks, there were little tasks that I just hadn't anticipated, and most things took longer than I had planned. By the end of Day 7 (my buffer day), I was done with the tasks I had planned for Day 3 and had a tiny start on Day 4, which is to say that I'd completed the decking, hadn't started on the railings or framing, and had one wall of the fort framed, but that was all.

DAYS 7 AND FOLLOWING

Since I'd used up my planned full-time days on Day 7, the rest of the fort had to be completed after work, so I had to work on it only a few hours at a time, and I couldn't work on it every day. So my calendar time overrun started stretching out faster than my effort overrun did.

OVERALL RESULTS

My original plan had called for about a week full time and then another couple of weeks of finishing up loose ends like painting, installing trim, and so on. I finished the fort about 6 weeks after I started it, so I was about 100% over my planned schedule, and I ended up at 2-3x my originally planned effort. Here are some pictures of how it turned out:

ESTIMATION LESSONS LEARNED

I mentioned some of the specific estimation mistakes on Days 1 & 2. As I got into the project I noticed several other issues that the estimates I'd made for my project had in common with errors in software estimates that we see with our clients:

1. Numerous unplanned problems collectively added up. Here are a few of the problems I ran into:

• When I opened my chainsaw case, which I hadn't used in a couple of years, I found that the oil plug hadn't been screwed in tightly, and the chainsaw was covered in oil, as was the case itself. I had to clean up the chainsaw and then dispose of the oil.
• When I was digging the holes for the footings, I chopped my layout strings a couple times with the post hole digger. I had to spend time repairing the strings and making sure that everything was still square.
• I dropped a little piece of my laser level down the side of one of the footing holes, between the concrete form and the dirt, after I'd poured the concrete. The piece was perched about 8" into the hole, just where I couldn't reach it. If I touched it wrong, it would drop the full 24" to the bottom of the hole where there was no way I could retrieve it. So it took me awhile to figure out how to get the piece out without risking losing it altogether.
• My jigsaw has a little compartment/drawer to put spare blades in, and it kept coming loose and spilling blades on the ground. I looked at it several times before the sunlight finally hit the opening just right to see that there was a blade stuck under the slot where the drawer is supposed to slide in that was preventing the drawer from going in quite right. Getting that blade out took about half an hour.
• I had trouble stabilizing the deck-railing posts by the "drawbridge" (gate). In hindsight, I should have used double joists on that side of the deck, but I didn't. I spent a lot of time staring at the these two posts, wiggling them, adding blocking to the joists below, and so on.
• There was no adequate footing for a ladder on the backside of the fort, and there was a big tree that made putting a ladder in awkward. Tasks that took 10 minutes on the front of the fort (like nailing up a facia board under the roof) took an hour on the back.

I think these problems are EXACTLY like the kinds of problems that show up unexpectedly on software projects -- two new tools you buy don't interface with each other like they're supposed to, and you have to figure out why. Or you install a new tool and suddenly your code doesn't compile anymore. Or you have a module that keeps producing errors because the design isn't quite right; you think you can't justify completely redesigning and rewriting it, but you end up nickling and dimeing your way to a higher cost than if you had bitten the bullet and redesigned and rewritten it.

2. Underestimation of unfamiliar tasks. My estimates weren't too far off for a lot of the work that I'd done before. But some things, like mapping out the site for the footing holes, I assumed would be 15-30 minute task ended up taking several hours.

3. Not decomposing big tasks into smaller subtasks. I'd planned out my project in whole days. At a birds eye view nothing seems obviously wrong with planning "frame the fort in one day." But when you break it down and say, What's involved in each of the 4 walls, and then you realize that one wall includes a door, another wall includes an angled top plate, a third wall includes an angled top plate and a window, and so on, and then you think about what's involved in each one (measuring, cutting, checking for square, recutting anything that wasn't quite right, tilting the wall up, checking again for plumb and square, attaching and then removing the temporary supports, etc.), you start thinking, can I really do a whole wall in 2 hours? If the answer's even close to "no," then you start to realize that the whole estimate for that big task is probably wrong.

3. Using overly round time units.  Using round units like "1 day" contributes to not thinking hard enough about decomposing large tasks into smaller tasks.

4. Substituting a target for an estimate. I had 7 days to do the project, and my estimate turned out to be 7 days. That's a little suspicious, and I should have known better than to make that particular mistake!

5. Sweeping numerous little tasks under the estimation rug. There were lots of tasks that I knew needed to be done, but I didn't want to admit that they were going to affect my schedule, and so I tried not to think about them or I minimized their impact (I realized this later). These tasks ranged from clearing brush from the site, to painting the trim, to installing the door knob. I think this is strictly similar to software estimates in which people just don't want to acknowledge that data conversion takes time, installing new tools takes time, converging each release takes time, etc., etc.

6. Never creating a real estimate. The fact of the matter is that I carried around a rough plan in my head for weeks, but I never actually committed a schedule to paper, and I never even considered creating a detailed estimate for the project. Of course the likelihood of making the other estimation mistakes I mentioned is higher when you don't officially create an estimate!

7. All's Well That Ends Well. My kids love their fort, and I had a great time building it. "All's well that ends well" is one reason that companies don't improve their software practices more often than they do. If people like the software that the team produced, and the software is successful, then that reduces the incentive to try to do better next time.

Some Differences

There were a few differences between my fort experience and a typical commercial software project:

• There was no way I was going to compromise quality for the sake of schedule. I couldn't build something that would be hazardous to my kids or their friends. So the schedule was going to be whatever it was going to be -- it was clearly a secondary priority. We don't see many companies in which quality trumps schedule to the degree it did on this project.
• My schedule overrun was free. My extra time was essentially free on this project -- maybe even a bonus since I was enjoying the project. So my overrun didn't imply a cost penalty. On a software project, you'd be paying for extra staff time, and so you'd have a cost overrun along with the schedule overrun.
• The estimation error didn't really matter, because I was going to do the project regardless of what the estimate turned out to be. If I had created a real estimate and had learned that the project was going to take 100 hours instead of 50 hours, I would still have done the project. It's much harder to justify not estimating and then flying blind for a business project, especially in the era of Sarbanes Oxley.

What Do You Think?

Are there other lessons I should have learned? Are these the right lessons? Are these parallels between fort building and software estimation valid at all? Let me know what you think.

One of my readers asked the following very reasonable question: 

We are looking for industry benchmarks detailing the amount of time developers spend on a percentage basis in the following three categories:

1) Core job activities (writing, testing, deploying code, etc.)
2) Meetings
3) Administrative activities (training, reporting, etc.)

The questions are reasonable. Unfortunately, one of the lessons I've learned after looking at lots of data on questions like this is that sometimes reasonable questions don't have reasonable answers!

In this case, what I would call "project focused hours" per month can easily vary by a factor of two between different companies based on factors like how much time is spent in meetings, how long the work days are (think government job vs. internet startup), number of holidays, number of training days, number of non-project meetings, level of support required for software already in production, etc. A common "big company" planning number is 6 hours of project-focused work per day, for the days that the employee is actually at work, but that can vary a lot across big companies and even within big companies. Based on what we see in our consulting practice, I think it's rare to average 6 hours per day of truly project-focused work in a non-startup company. The most common distraction from project-focused work we see is time spent supporting prior releases that are in production.

The number of meetings varies a lot too and is significantly affected by company culture. When I was at Microsoft in 1990-91 I probably spent less than 5 hours a week in meetings. In contrast, I had a former Microsoft employee tell me earlier this year that on the team he was on he was booked in meetings from 10:00-4:00 5 days a week. Lots of managers at other companies have told me that they're in meetings all day every day and get most of their "real work" done during evenings and weekends, so obviously there's a big difference between Microsoft 1990 and Microsoft 2007, and among different companies.

The amount of training, reporting, etc. varies just as much--it varies even more on a percentage basis. Best in class companies typically devote 8-12 days per year to training, whereas many companies we see allow technical staff to take 1 class per year. Many of the companies we see don't systematically support any training days per year. 

Bottom line is that there's just too much variation among companies to make meaningful statements about "benchmark" allocations to work and overhead time categories. That doesn't mean that you won't find published sources that claim to be benchmarks, but if you do those sources are usually limited by the fact that the authors haven't had exposure to enough companies to realize that there's as much variation as there is.

Resources

• Software Estimation: Demysti