Project Management acts like a Marketplace for Information

“A social problem!?” Really? This is painfully obvious. Most managers do try to juggle the social issues of a software development project with the technical issues. Clearly this is not news, clearly people understand the implications of social issue son software development.

And yet, there are two social issues that are often overlooked in project management. In this chapter, we are going to look at one of them, The marketplace for information and its effects on development projects.

Information is the raw material used in every software development activity. Developers turn ideas into code (with caffeine acting as a catalyst, of course), product managers turn problems and opportunities into features, and project managers turn information about the state of the project into a to-do list for what to work on next.

Pipes

Close your eyes and imagine a development team as a big steam engine. There are boilers, valves, tanks, turbines, pistons, and of course pipes. The pipes don’t do any of the work. But they do the vital job of moving the water and steam around to where it’s needed.

Metaphors are leaky abstractions. So no, information is not like water or steam. But the image of pipes helps me remember how important it is for a development project to convey information within itself without losing or corrupting it along the way.

The trouble is, there is another metaphor for the behaviour of information within a development project: A marketplace.

There is a market for information in a project, with information flowing to where it’s valued and away from where it is not valued. This would be fine if the “value” of information was always its utility for moving the project towards completion. However, markets for information are just like “real-world” markets: Full of uncertainty and conflicting human needs. They are inefficient.

In order for information to flow to where it is most valuable for moving the project forward, the participants in the project must value the information properly. Managers “buy” information. They trade favours like letting you keep your job for information about how well you are doing your job. In order for them to buy it, they have to “price” it.

Fruit Market

Alas, most managers, especially those with limited experience shipping software on a predictable schedule, do not know how to correlate what they’re told about the project with the likelihood that the project will succeed. Thus, they cannot properly price the information.

When they price it wrong, they pay high prices for junk information–like whether a developer is punching the clock consistently or how many lines of code they wrote–and low prices for valuable information–like the thought that a massive infrastructure investment might be avoided if the team pushes back on what seems like an inessential requirement.

Managers also “sell” information, literally: They have to make a report or a presentation to their superiors, or to stake holders, or to their fellow founders at the YCombinator dinner.

When a manager cannot tell the difference between information that is useful for predicting the outcome of a project and information that is not useful for predicting the outcome of a project, she thinks about the next best thing: The “resale value” of the information with people one step removed from the project, like her own manager. So she values things like pretty PowerPoints about the architecture higher than finished pieces of functionality.

(This is why I have always sweated my heart out to give good presentations. My teams have depended on me being able to take good information and sell it upstream just as if it were CMM Level Five Buzzword-Compliant Junk).

Do managers further removed from a project always value pretty junk more than good, solid information? Not always, but often. And that’s enough for people to be pressured to give the bad information that sells to their manager, while hiding the good information that doesn’t sell. Exactly like the owners of good cars taking their treasures off the market.

So the lesson here is that while project managers can directly use information, they sell information, and many times the “resale value” of the information is more important than its intrinsic value to the manager. This creates an incentive for them to buy information with high resale value regardless of its intrinsic value to the problem of shipping the software.

what kind of information sells?

Why does junk information outsell good information? Nice PowerPoint isn’t a good explanation by itself: there are nice PowerPoints explaining Agile, but most managers still prefer Waterfall.

Consider a “Not-So-Big” design. Let’s call completing that design good information: we’ve done a good job finding out what’s really important for the project and making a design that emphasizes the way this project is unique, not the technology stack.

Now consider a typical technology design, emphasizing frameworks and technologies. Fully buzzword-compliant.

Which one sells better? The technology stack does. Why? Well, for starters, managers have been exposed to seventeen billion dollars worth of advertising talking about the benefits of technology stacks. Nobody is advertising the specific ways the not so big design helps the project. How could they? Those are specific to the project, that’s the whole point.

And managers are like anyone else, they compare what you are doing to successful projects they have seen in the past. Once again, the not so big design doesn’t have anything in common with other projects, but the technology stack does. (There are lots of failed projects with technology stacks, of course. But who cites those when bugging the team about whether they will use Hibernate as their ORM?)

How did this happen? How did things that have no correlation to the success of a project become more attractive than things that do?

Quite simply, people have an incentive to look successful. So they imitate the outward appearances of successful projects. We have a really simple way of completing successful software projects: we put successful people on them. But we have a broken way of thinking about it: we don’t like to think of the people as being special, we think that what the people do is successful.

And by that logic, we can take anyone, have them do the same things as successful people, and our projects will succeed. In a manager’s mind, the measure of whether information is good or not is, Does it measure whether people are doing the same things that successful people have done on projects I’ve been told were successful?

This is not the same thing as measuring whether the project is on its way to success at all. This measures the outward appearance of a project. Things that can be measured easily are rarely the most significant things. Behaviours that can be “gamed,” like how many hours a team is working, will be gamed.

And as above, even if a manger knows better, does her manager know better? If not, good information will be difficult to sell and she will be under a lot of pressure to come up with information that does sell, whether it’s good, indifferent, or even harms the project.

The marketplace for information has a profound effect on the viability of a development project. There are natural incentives in nearly all social organizations for bad information to outsell good information, and for that reason, a key to success in project management is the ability to recognize the existence of this marketplace and ruthlessly work to ensure that the good information is conveyed within the project and the bad information is forced out.

(This chapter adapts a portion of “Still Failing, Still Learning,” originally published in June, 2007. The pipes image can be found at http://www.flickr.com/photos/seeweb/6115445165/in/photostream/ and the fruit market image at http://www.flickr.com/photos/xavitalleda/6168935426/in/photostream/.)