23 October 2003
“SHOPPING” FOR RISK MITIGATION

by Michael Mah, Senior Consultant, Cutter Consortium

“Trends in Risk Management.” That was one of the topics of a lively discussion following Cutter Business Technology Council Fellows Tim Lister and Tom DeMarco’s keynote on risk management at the sellout Cutter Summit 2003 conference.

[If you weren’t at last year’s Summit, you should have been. Or you can get the audio CD. Better yet, reserve your place for Summit 2004 before the inn is full. And by the way, buy Tim’s and Tom’s book on risk management, Waltzing with Bears: Managing Risk on Software Projects. End of shameless commercial. I’m practicing what I preach. I enjoyed Tom’s and Tim’s keynote so much that I bought copies of the CD and sent them to all my favorite clients (you know who you are; enjoy the gift in your car, at home on your stereo, in your discman…).]

Tim and Tom’s keynote highlighted how increasingly important risk management has become in corporate IT organizations. Why? Because of the increasing proliferation of software and IT in modern society and the increasingly high stakes that are involved in today’s projects.

An example of this was their discussion on the lessons learned from the now infamous Denver International Airport baggage-handling project (yes, the software was late). But for me, the kicker of the debate was something that Tim said. Not everyone may have noticed, but I did. In essence, Tim said, “Most of the time, the deadline is dictated to you. You don’t always have control of the number of people — or their experience levels — on your team. So what’s left that you have to control?” Answer: “The amount of stuff that you’re stuck with to build! So build as little as you have to. Make every line of code count!”

So, to my simple mind, this is what we have to negotiate. Sign up to do too much stuff — and you’re in big trouble. You need to sign up for less stuff. How much? The task at hand is to find out how much you can take on before an imaginary robot pops up on your shoulder and says, “Danger Will Robinson!” (Darn. Him again.)

I have a client; a really important client — a major international financial services company. These folks do things in a big way. There’s nothing small about what they do. Their projects are big, the stakes are high, and the deadlines are, shall we say, aggressive.

Anyway, they’ve got this big project right now. We’ll call it “Global Monster Financial Project,” or GMFP. They just accomplished a most remarkable success. I’m quite proud of them. It will take their risk management practice to an entirely new level if they stick to their guns. As Emeril would say, “They really kicked it up a notch.”

But it started with a missed date, and it was bad: GMFP Release 1.0 was late, and not by a little (no surprise really, all 1.0 releases are late). But this cost BIG TIME — maybe $1.75 million a month for six months. Do the math on the overrun: 10 million dollars. Oops. (How much is a software project estimating tool? Sold. We’ll take 20, please.)

Anyway, here’s what I’m proud of. Not the overrun, but what the release manager (who came onboard after 1.0) did after the overrun. On future releases, they have deadlines. They basically have a fixed headcount. So every release comes down to negotiating the scope or the functionality with the client organization. That’s it. The weekly question is: “How much can I buy (that you can build) in, say, eight months?”

So we put our thinking caps on, worked long hours throughout the summer — because the next FY 2004 budget planning needed answers (a couple hundred million dollars’ worth of answers) — and we came up with … a shopping cart.

Not any old shopping cart. This cart lists all the types of components that make up the building blocks that will comprise future GMFP releases. “You want 75 ‘Complex Reports’? Check. Twenty-six ‘Extract Transfer Loads’? Check. Fifteen ‘Unix Shell Script Modules’? Check. Tally the shopping cart all up, and….

Then you hit this really cool button that we tagged “Proceed to Checkout.” Next, the computer tallies all the components in the cart. Each component takes a certain amount of programming actions. Some is code. Some isn’t. The latter involves things like configuring data fields for users and implementing settings in a purchased package. They’re “logical instructions” or “implementation units.” Anyway, it’s like currency. You add up all the items in the cart, and you find out how many programming actions or implementation units you need to create that component. Some items in the cart are the same size. Others might be two or four times bigger than other items. Why does this work so well for GMFP? Because they feel (rightly so) that neither lines of code nor function points are a good fit in their environment, and they want a better alternative. (Another casualty of the function point religious wars.)

The computer then bounds a degree of uncertainty, say, plus or minus X%, based on how confident you are on the individual quantities you need. It tallies all this up and when you click “Proceed to Checkout,” it dumps the data into a calibrated estimating model. You say, “I’ve got 25 people available. We know they can build GMFP financial stuff at this Productivity Index. You get a Monte-Carlo Simulation of how much it will cost. All the Gantt charts and milestones forecasted; tailored to the GMFP lifecycle template. An entire staffing profile for all the major phases. And (drumroll, please) the date when the release will be delivered. (All spooled to make a Powerpoint presentation if you like.)

Now, what if the date isn’t what the Big Cheese Boss demanded? Well, here’s where the fun begins. If the date can’t move, you’ve got to tell the boss how much the team can build. We go back and run “war games” simulations on the computer. We try out a bigger team. Sorry, don’t have 50 people. We try different productivity assumptions. Hmmmm. No one on the entire planet has ever achieved that before.

In the end, it comes down to what I call, “deadline-driven estimation.” You put in the date, you dial in the number of people available, you bound the limits of productivity that they can possibly achieve, and then you click a button. The answer comes. It basically tells you how much you can fit in your cart. It’s similar to when you go into a store and say, “I’ve got 20 bucks. What can I buy?”

And here’s where the conversation simplifies. It all comes down to realism on how much functionality a team commits to on its promise. You can even go to a risk diagram showing an S-curve that answers the question “How much can I deliver and have a 90% probability of making it?” This is the focus of the negotiations. We brainstorm the options that have merit, put them all on the table, and try to see how everyone can be happy.

That’s risk management in action. But hey, it gets better. The project starts. Guess what? Things change. What we want in the cart is different, now that we know more. The client’s needs shifted. We outsourced that piece; now we need to build another piece. Well, there’s the equivalent of an air traffic control tower that’s been tracking basic metrics on milestones, effort, size, and defects for the releases “in-flight.” It’s a computer model that’s been displaying either a green light, yellow light, or red light all along, telling us if a release is off course. And the cool part is, any change in flight plans can be simulated on the computer before yanking the controls to the left or right. (Keeps projects from crashing.)

The moral of the story: I sometimes get impatient when people just talk about risk management. But it’s really exciting when people actually do it. And let’s face it, there’s nothing to be ashamed of when a project slips and we get hit with a budget overrun. Big deal. It’s been happening since the beginning of time. What’s important is this: If everyone were to act similarly in response to project overruns, there would be a broader adoption of sound risk management practices that, in total, would truly change the direction of our industry. It’s the things we learn, what we do in response, and how we get better that counts. GMFP Release 2.0 is going to be a lot better. Because smart people got off their duffs, stopped wringing their hands about what they should do about overruns and slippages, and actually DID something about it. Bravo!

— Michael Mah, Senior Consultant, Cutter Consortium

© 2003 Cutter Consortium. All rights reserved.