14 October 2004

by Michael Mah, Senior Consultant, Cutter Consortium

“If you want higher quality, build less stuff.” That, in essence, is what Cutter Business Technology Council Fellow Tom DeMarco once said about a daring strategy for quality improvement: reduce quantity. Tom went on, “Whatever it is that your organization makes, make less of it. Make less, and choose much more carefully what you make.” (For more on this, read Chapter 16 of DeMarco’s Slack, Getting Past Burnout, Busywork, and the Myth of Total Efficiency.)

The good news is, Tom’s advice is something that can be done immediately to reduce defects. Talk about being agile! Quality is inversely proportional to quantity — simple as that.

But what exactly is software quality? What is a “bug?” If your software operates a medical device like a radiation therapy machine to treat cancer, a bug could mean real harm to patients. If your software operates a ground-to-air transponder that sends low-altitude warnings to aircraft during takeoffs or landings, a bug could result in a tragic accident.

Maybe your software isn’t safety-critical. It might be something like a telecom switching system. In this case, a bug could start a domino cascade that shuts down your entire East Coast network. Or if it operates a computerized stock trading system, you might have an outage resulting in millions of lost financial transactions.

All of the above examples really happened. In the case of the medical accelerator, it’s believed that a bug in the code caused radiation doses to be multiplied by 10E6 (10 followed by 6 zeros) instead of plain old 6. One patient undergoing radiation following a lumpectomy received 15,000 to 20,000 rads. Typical doses are 200 rads, while a whole body dose of 500 rads is considered fatal. In the end, the machine — known as the Therac-25 — caused death and serious injury to six people. In the case of the ground-to-air transponder, the fact that it had to be taken off line is believed to have been a contributing factor to a 747 tragically crashing in Guam with 231 people on board while attempting to land.

At present, several of my clients are trying to build too much code within too tight a deadline, with quality suffering as a result. Projects grow large, code slips because of requirements changes, and near the end, there’s simply not enough time to test — regardless of how many overtime hours the stressed-out team works. When the deadline arrives, there’s great management pressure to ship, and the bugs rule. Two are medical systems firms, one a telecom company, and a fourth is a Web-based financial services firm.

Many of my clients’ teams struggle with software project estimation. In some cases, productivity is high, but quality is low. On several of these projects, I helped teams benchmark the defects found during systems tests. It turned out that the number was low compared to the industry average (from an historical database — a “Kelley Blue Book” of modern software projects). That seemed odd at first. But then, we looked at the defects in the first month of operation: way high compared to the industry trend! What we were seeing was a bug curve pushed way to the right on the timeline. Bugs during testing were low because they got a late start. More bugs showed up to the right of the deadline and not the left, all because they tried to do too much.

A lot of this is the fault of what I call the “Productivity Paradox” — that more is better. In software design and other forms of knowledge work, more can indeed be worse. Culturally, we are stuck in this mindset. We simply take on too much. A Cutter Consortium study showed that when projects miss their deadlines, 72% of the time it’s because of scope growth.

That’s why we’re also obsessed with cheaper labor rates for offshore programmers: more for less. What if I told you that benchmark statistics from an international database of 7,000+ projects shows defect rates for offshore projects being 35%-40% higher compared to domestic projects? What’s better? Does this software fly airplanes or operate medical devices? If so, be very careful. I once helped a client evaluate an offshore proposal where the project was indeed a very low fixed price. But the fine print said that warranty was short and that bug fixes were extra. My client decided to keep the project in North Carolina.

In the short term, give up a little productivity, for quality’s sake. To do this, you’ll need to learn how to manage requirements and estimate projects in a different way. Given a fixed deadline and a team of, say, 15 people, your challenge will be to forecast how much functionality you can build — and not have it break 10 times a day after you ship. This is called “deadline-driven estimation.” There are even commercially available estimation tools that do this for you.

You’ll also have to polish your skills on sizing your projects and negotiation, because your customer will want more and more, within ever tighter deadlines. They’re stuck in the productivity paradox too. Plus, negotiating scope will require more advanced skills than you’re used to — this isn’t in the same league as haggling the lowest price at a street market. Information technology is about design work, and that complicates things.

Why are things more complicated? Because what we design has to work! We also are faced with a multidimensional negotiation — scope, staff, cost, and quality. If we’re not constantly cutting corners because we’re overextended on scope, we have a better chance of coming up with what could be a beautiful design that works flawlessly, makes our customers happy, gives us a sense of personal and professional satisfaction, and makes everyone proud. Easing up on the 70- to 80-hour work weeks might also give us more time to parent our children, spend time with our partners, and lower the divorce rate.

So choose what you build carefully, then build a little less of it and decide to build it very well.

— Michael Mah, Senior Consultant, Cutter Consortium

© 2004 Cutter Consortium. All rights reserved.