Man Vans, Speed, Technical Debt, and You
I have what some people call a “Man Van.” It has 4WD, alloy wheels, big ice-grip snow tires, tinted glass, and a 200-watt stereo. David E. Davis, past editor of Automobile Magazine, once wrote that his Man Van was his favorite car, even compared to his Ferrari and Dodge Viper. I’m no soccer mom, but as a single father of two, I can haul dorm-room piles of student belongings up and down the US East Coast. Most important, in a snow or sleet storm, I can drag-race anyone and leave them eating my dust … um, snow.
The Man Van is large and heavy, like a large software team. It gets decent fuel economy unless you try to push it hard and fast, which is when wind resistance takes its toll. I once pushed the Man Van to 90 miles per hour on the highway, glancing at the radar detector at the slightest beep. Sure enough, my fuel economy degraded dramatically. Trying to go even faster had a diminishing rate of return as wind resistance increased. (Don’t try this at home, folks.)
Trying to push a large software team fast also hits a compressible limit. Recently, I consulted on a high-pressure project with a tight deadline. To go really fast, management ramped the team up to 70 people. Despite this giant offshore and onshore hoard, the schedules refused to shorten beyond a certain point, and the team began missing its dates. It reached what we call “terminal velocity” that defined the minimum development time. All projects have a minimum development time. It’s where wind resistance matches the forward driving force of the team.
What determines this minimum development time? Obviously, the first is project size and scope. If it’s an absolute truth that to deliver 82 features requires six months — no less — then wishing you can deliver in four months is pure fantasy. “Reality will prevail,” as I like to say — plain and simple.
Another driver is productivity. In agile development, this is often referred to as “velocity.” To me, I see it more as the “innate productivity” of a team actually results in a certain velocity. All organizations have an upper limit on productivity. The problem is that most organizations have never measured this and don’t know what it is. It’s driven by several factors, such as the experience of the team, the development approach, and the natural complexity of the task at hand — all acting in combination. These and other factors limit productivity to some upper threshold. It’s there, whether or not you know it.
Then there’s a third factor at play, and this is related to wind resistance. It has to do with the limits of large teams and what happens to defects as you ramp up staff in order to compress time. Ask anyone in a room whether defects tend to go down or go up as you try to compress time with large teams. The answer is that defects will rise. What most people don’t realize, based on research on thousands of projects, is that if you double team size, say from 35 people to 70 people, then you get a geometric rise in defects. It’s typical that they’ll rise by as much as 400%. In the worst case, I’ve seen numbers climb as high as 600%. This is known as the Fourth Power Tradeoff Law, discovered by industry pioneer Larry Putnam, Sr., and built into a project database called the QSM SLIM model. It’s the result of the geometric rise in miscommunication as more and more people are thrown on a project under time pressure. Haste makes waste. (Good software estimation and forecasting models can help you avoid waste.)
When the deadline finally arrives and a software release isn’t ready, it sometimes comes down to a terrible choice: shipping with lots of bugs, or slipping the date and spending more money. Given that choice, many execs often say two tragic words: ship it. BAM! Instant technical debt. (Let’s just hope that the software isn’t meant to fly airplanes or run other safety-critical systems.)
It’s this defect factor that acts like wind resistance to limit the speed of a large software team. It gets even worse when large teams aren’t co-located or are spread across far-away time zones and long distances. Aside from miscommunication due to giant-sized teams, you get mistakes from language barriers or thick accents, or fatigue when it’s midnight for you and noon for them. One manager in Germany told me that, when speaking to a team in Singapore, people were attempting to understand each other on complex problems in English, a language that was foreign to all of them. They couldn’t communicate well, and critical missteps killed the project.
Relentless pressure to make a date starts a vicious cycle of technical debt, which includes latent defects in production software that eventually must be corrected (e.g., paid down), else the business suffers in the form of potential software outages, poor performance, and/or a host of other consequences of buggy software. If this debt is not paid down due to neglect in software maintenance, technical debt simply accrues. Finally, at some point, this debt begins to crush the business, and diversion of critical staff resources toward repairing production code is needed.
Where do these staff resources come from? Obviously, from skilled developers in the company who know the systems and architecture the best. However, diverting their efforts to paying down technical debt diminishes velocity and productivity for developing new applications. Relentless schedule pressure continues, projects fall short, and the vicious cycle continues.
So what’s the answer? Well, it sounds simple: in an ideal world, you run a project with the smartest people you can find, sitting colocated to the greatest extent possible, while planning and estimating the scope of the project within the limits of that team’s productivity. That kind of team, if you can get it, responds quickly to change, builds cleaner code, and finishes sooner because they finish testing faster, since there are far fewer bugs in the first place. Another potential answer is in using agile methods to mitigate the communication complexity imposed by the conditions cited above. While no panacea, benchmarking research studies that show that the introduction of high-bandwidth communication techniques espoused by agile can often mitigate these risks.
Or you can push a big brick-shaped Man Van against the wind, burn lots of fuel, and risk crashing it if you push it too far beyond its limits.
You are welcome to share your thoughts and impressions about this topic via comments on this blog post.
— Michael Mah, Benchmark Practice Director, Cutter Consortium