June 4, 2007

http://www.projectsatwork.com/content/articles/236756.cfm

The world may be flattening, but it’s also clustering, as I described in this interview that I gave for projectsatwork.com. In this, I make the case for the “energy synchronization” of co-located project teams, describe why creativity and outsourcing are difficult bedfellows, why adding staff often exacerbates problems on high-pressure projects, and how offshore, in-house and agile-led projects fare on defect rates. You can read the full text at the projectsatwork website, but I’ve also included the questions and answer section here. Enjoy!

==========================

Five Questions – Agile Methods and Outsourcing in a Flat World

Is the world really as “flat” as proponents of Thomas Friedman would have us believe?

Yes and no. It’s true that globalization has enabled all kinds of work to become decentralized and spread across the globe, with workers being split geographically across 2 or more continents.

But on the other hand, people around the world are “clustering” together more than ever before. Fifty percent of the world now lives in urban areas. There are “megacities” like Seoul, New York, Mumbai, Shanghai, and others that boast 20 million or more people.

There’s a reason for this, according to people like the Nobel-prize winning economist Robert Lucas and George Mason University professor Richard Florida. It has to do with the powerful productivity gains that come from the “energy synchronization” that occurs when teams of people are co-located. You can feel this yourself when you’re with an enthusiastic person, working together on a new idea. Creativity flows, your excitement feeds off of one another, often both in and outside of work, and it’s more fun.

So the world is flat, but it’s also “spiky,” with clusters of people around the world getting larger and larger as rural populations migrate to urban areas. It’s actually creating a crisis in the countryside in places like rural China.

In the world of technology, this is coming down to two forces at play – the continued outsourcing and offshoring of technology work, but with increasing competition from people arguing the case for work being done in-house, using methods like Agile software development. The former argues to split people up, and the latter aims to bring them closer together. My work involves measuring and managing the outcomes of both approaches.
What are the primary forces that are driving outsourcing?

One primary force: the lowering of costs by using cheaper labor. We’re seeing more “forced outsourcing” when it’s dictated by the executives like the CFO and not the head of engineering or product development. It seems less about accessing skilled talent overseas and more about cutting costs. There’s plenty of talented people right here in the USA and North America, but if they can’t be hired at third-world wages, they’re sometimes out.

However, recent studies reveal that actual cost savings might be shrinking, and that it’s smaller than we thought in the first place. A survey by the Cutter Consortium of about 100 organizations showed that a typical IT organization was outsourcing about one-fourth of its work. On that one-fourth, they’re labor costs are about 20 percent less on average. So the net savings is 20 percent of the “one-fourth slice,” which comes out to five percent. That’s not a lot at the end of the day.

And unfortunately, we’re seeing that it’s hard to coordinate work across multiple continents and time-zones. Many projects slip their dates, and experience higher defect rates in spite of good intentions. That drives up costs. Teams often find it difficult it is to communicate with one another “through a wire,” and it sometimes drives the outcome of the project.

Isn’t there credence to the idea of a “software factory?” I thought you could take IT work and “chunk it, routinize it, digitize it, automate it, and then offshore it?”

Yes, some routine cognitive tasks can be done this way. It appears much harder when expert thinking, creative problem solving, and complex communication is involved. Simple maintenance tasks fall into the former. Innovation and cutting-edge software development “inventions,” fall into the latter.

In the words of Rob Austin, professor at Harvard Business School, “The future belongs to those who can create new things.” That is innovation and it is creative and non-repetitive. (Things like YouTube, Skype, and even iTunes are basically “killer software apps.” Remember that term? They represented new things – incredible innovations that no one had heard of before, and they blew the top off of the box in their respective markets.)

In manufacturing, we know what to do and how to do it. We automate the process. If we want to double the output or halve the time, we add another assembly line or speed up the machines. That is a factory, and lower labor rates are relevant.

However, when it comes to things that require creative expert thinking and complex communication, this applies: To some extent we don’t know what to do or how to do it. We spend most of our time, not building the system, but figuring out what to build and how to build it, by talking to one another. And adding more staff can have negative rather than positive consequences, because of communication complexity.

The difficulty some companies are finding with outsourcing is that “talking to one another” piece. Compounding the problem is talking through a wire when you’re fresh and they’re tired because of vast distances and time-zones, or talking when you’re tired and they’re feeling fresh.

The idea of a software factory tries to force a cost-cutting Frederic Taylor manufacturing philosophy – into a creative economy, killer-app, innovation culture. It’s totally paradoxical, and while we don’t hear it used by name very much these days, in practice the idea is still out there in force.

In software, the idea of “finishing the requirements” and handing them over to low-cost factory coders in another country is extremely rare, if nonexistent. In truth, during the design and build phase, the requirements constantly churn, often requiring lots of complex communication and feedback loops. Agile methods by design are intended to meet this challenge head-on.
What is industry productivity data showing on projects that are offshored, versus those that use Agile methods?

We have statistics on several thousand IT projects right up to the present. We’ve seen productivity and quality patterns on in-house projects, offshore outsourced projects, and now, patterns on agile projects are emerging.

We are seeing similar productivity numbers. But not surprisingly, the defect patterns are different. When offshore projects experience that “talking to one another” problem I referred to earlier, the bug rates are higher. Sometimes 2x to 3x higher. When agile projects successfully execute a co-located team strategy, the defects rates are 20 to 50 percent lower.
Is it possible to successfully outsource to an offshore vendor who uses Agile methods?

That’s a real tough one to answer. Let’s think about it. By definition, an agile method like Extreme Programming (XP) means “co-located teams.” Offshoring, by definition means splitting teams, often across 2 or more continents and multiple time zones. If someone can make those two definitions co-exist simultaneously, it would be a remarkable accomplishment.

I’m not sure that it can follow the “Reese’s Peanut Butter Cup” recipe – you’ve got chocolate in my peanut butter, or peanut butter in my chocolate. Software engineering might be more complicated than taking two different flavors and mixing them up.

But we don’t have to worry about the answer – people are bound to try it, and if they do, we can measure it. A few companies that I’m working with are attempting it. Stay tuned, the results are bound to be intriguing.

Michael Mah