Managing Product Development

Earlier, I made an off-hand comment, “The zeroth measure of productivity is showing up.” I now think showing up is necessary, but not sufficient. I’ve been thinking about what each of us produces individually, and thinking of ways to understand and possibly measure it:

1. How many hours per day do you stay focused on strategically important work? Sometimes, I keep a time log to make sure I’m working on appropriate things. Especially if my to-do list is very long. Like many other people, I can be distracted by things that are easy to do, but not as necessary as the work that may be harder to finish.
2. How many stupid mistakes do you make? When I was younger, I was so proud of myself. I would take myself home after making three stupid mistakes. ARGH. Three! Why did I have to make more than one stupid mistake??? I learned that three stupid mistakes is two mistakes too many. Now, if I make my mistake at 9am, then I can clean up email or my office for a few hours until my brain has a chance to think again. If I make another mistake, I take the rest of the day off (from hard-thinking work). There’s almost always something I can do that doesn’t require hard thinking.
3. How many of what did you accomplish in a week? How many times did you revisit that work? When I’m writing, sometimes it takes me too many drafts to find the right way to discuss the topic. Same for creating presentations. When I was a manager inside organizations, I noted how many decisions I had to re-open. When I was a tester or developer, I looked at the work to see how much rework I had to do. Finishing a piece of work the first time isn’t enough; the work isn’t complete until the people you hand the work off to agree that it’s complete.

So, my measure of personal productivity is how long I can stay focused on useful work, and how few times it takes for me to accomplish that work until the person I hand off to is satisfied.

The zeroth measure of productivity is showing up. Sorry I haven’t been showing up, but I have to admit, sleeping is good Ok, back to business.

When I tried to calculate productivity of a team, here’s what I measured for one team over the course of five releases (Apologies to bloglet subscribers; tables don’t email well):

Ok, so this team had a number of problems. The first problem was that they didn’t think defects was part of what they produced, so they didn’t count defects for the first two releases. By the time they started counting defects, the defects overwhelmed them.

Their management didn’t think counting defects was important at first, which is why their apparent productivity was so high for the first release, and not too bad for the second release. By the time they attempted release 4, the product was so error-ridden, they couldn’t make any progress on features, and were adding to the bloat and defect levels. For release 5, they instituted peer reviews and better end-to-end testing, so they could see if their fixes worked.

Contrast this traditional development with more agile projects. In an agile project, you can determine each iteration’s velocity (number of user stories per iteration) and the number of outstanding defects. Because people on agile projects tend to perform test-first development the sheer number of defects tends to be fewer. And, if the testers and the customers write user acceptance tests in advance of the first line of code, agile projects tend to have fewer egregious defects. (The one problem I don’t know how to solve with agile projects is keeping the customer involved.)

So if you’re measuring productivity, try measuring size, time, number of people, and defects. Once you’ve calculated some sort of team productivity, see if you can figure out personal productivity. Just don’t forget that a person’s productivity is based on their entire contribution to the product, not just the number of lines of code (or tests or whatever) they contribute.

When we think about manufacturing work, we measure labor productivity as the ratio of the output of goods and services to the labor hours devoted to the production of that output, output per hour. (See U.S. Dept of Labor)

Remember the discussion of Project Constraints and Requirements? That’s where I said the project requirements were a tradeoff of how much (feature set), when (time to market) and how good (defect levels). That’s the reason we can’t use output per hour as a software (or any knowledge worker) productivity measurement. Here’s why: If you don’t care how good a deliverable is, I can have it for you almost immediately. It won’t work, but my “productivity” if all you consider is when I say the deliverable is complete, is very high. (Not much time spent, so the ratio is high.)

If we want to start measuring developer or tester productivity, we have to define output per hour (or whatever time period you desire).

For developers, we can measure designs considered, lines of code generated, the number of defects generate along with the code, unit tests generated, unit tests run, how good that output is,and the time it took the developers to generate all that output.

For testers, we can measure test designs considered, number of tests generated, attempted, and run, test logs, defects detected, defects reported, number of measurements provided back to the developers as information,how good that output is, and all the time it took the testers to generate all that output.

Not so simple, eh? If we just knew how to measure how good our work was, we’d have a chance to measure individual producitivity. The more I think about productivity, the more I know it’s a project-by-project thing, not a person-by-person thing. Yes, some people have more effective output than others. And I’m not sure that matters.

If I can figure out how to share some data tomorrow, I will. Most of my data is under non-disclosure, so I have to be careful with what and how I talk about data.

In the past few weeks, too many managers have written to me, asking for help on knowing how “good” their people are. When I ask more questions, such as “What does good mean to you?”, they say they want to know who’s most productive. Then I walk through this analysis with them:

Productivity is not about the number of lines of code, the number of tests, the number of defects reported, or the number of requirements defined. Productivity is how long it takes the entire team to create a usable product, and for whom that product is usable.

It usually takes us a few emails to get past that statement. That’s because these managers now realize a single-dimension measurement is inadequate for measuring a person’s productivity, and that on a project, it’s the productivity of the project, not the productivity of a given person that matters.

For you lifecycle/process aficionados, that’s why agile projects, staged-delivery, evolutionary delivery lifecycles, and critical chain project management works. Each of those lifecycles or techniques focus on getting work out of the project, not waiting until the entire project is complete.

Tomorrow, I’ll deal with the kind of things you can measure.