Managing Product Development

I had the pleasure of speaking with two different colleagues today, both with the same dilemma. They are near the end of their projects. They don’t quite have enough time for one round of final testing–but if they’re lucky and the stars align, and they don’t find too many problems, they can still (maybe) test what they need to test before the desired release date.

But no, the stars don’t align. First week into testing, they find a nuisance of a defect. It only occurs once–in installation, and they can work around this with release notes–but they’re under pressure to make the change. They each asked me what I would do.

After asking a few questions to make sure the problem only occurs when installing, and they can make big red stickers to explain to the customers what to do, I agreed with the PMs: don’t make the change.

The risk is just too high. The reason the projects don’t quite have enough time for testing is that they’ve encountered trouble all the way through the projects. The builds take too long. The developers didn’t integrate testing as they developed. They implemented by architecture, and couldn’t manage the changes in requirements, so the architecture doesn’t quite fit what the customers want–but they shoehorned the features in anyway. The project team hasn’t met one deadline yet.

If you’re in this position with your project team, ask yourself and the team this question: What did you see or hear that leads you to believe this would work?

If the someone has data, “Oh, we fixed the build and we can tell in 10 minutes if the build is any good,” maybe you can agree with the decision to make the change.

But if all you hear is gut feelings, say no. Murphy is one of your team members. Finish this project. Hold a retrospective. Work differently on the next one. But don’t make that one small change. It won’t be small. It probably won’t work, and you won’t know until the customers receive the product. The risk is too great.

Labels: project management, risk

Scott points out Software Product Delivery - 20 Rules? that you should do the riskiest part of the project first. (He explains that you modify that given what’s most important.)

I’d add a further refinement: that what’s most important better provide the most value. If it doesn’t, do the most valuable parts first. You might not have to do the riskiest parts.

I saw this in action yesterday. I taught my pragmatic project management workshop, where part of the workshop is to do a project. The project takes somewhere from an hour to two to complete, and has a tricky architectural part. The idea is that PMs need to be able to see if the architecture is working.

One team got stuck on an architecture that cannot work–the resulting product is not stable under any circumstances. They came to me for more materials. I’d already given them more materials than they needed. With my customer hat on, I said, “If I give you more materials, how do I know I’m not throwing good money after bad? It doesn’t look to me as if this architecture will work.”

They were perturbed, to say the least. When we debriefed, they were still thinking that if I’d given them more materials, they would have succeeded. They were focused on the riskiest part of the project.

But if they’d thought about what was most valuable, I bet they would have developed another architecture–and they would have been successful.

Sometimes, release criteria can help you articulate what’s most valuable. Sometimes you have to ask someone. But starting with the riskiest part of the project may not actually solve the value problem. If you start with what’s most valuable, you’re more likely to succeed.

So after I posted the Probabilistic Scheduling post, I was working merrily away. I had made some small progress on the book, but was still finishing up other things. Finally, Wednesday I had cleared the entire day to work on the book. I was having trouble with one chapter, so I decided to make tea and do some timed writing. But I encountered an unanticipated event.

While picking up my electric teakettle, I fell down. Have no idea why, just fell. Normally, an ankle or knee gives out, and I fall sideways. I know to relax and go with the flow so I don’t damage joints worse than they are. But this time I fell straight down. Did a Greg Louganis on the table in my office.

Head wounds hurt. Almost as much as childbirth. And when they bleed, they bleed a lot. Called the doctor, breathing through the pain, was told, “Go to the ER.” Ok, got in the car, drove myself to the emergency room, waited several hours for my 4 stitches, and returned home.

I knew I was not myself; I didn’t even read in the ER. (I read all the time. Everywhere. Unless I’m with other people. But breakfast doesn’t count.) But by the time I returned home, I was in much better shape than when I left. The local painkiller was still working. I was no longer bleeding. And my headache was gone because of the local. I was still shook up, but certainly ok.

So, Wednesday was a fairly lost cause, at least for writing. I made more progress yesterday, and hope to do more writing this afternoon, after I take care of other little things. And yes, I’m fine. I have blue stitches on the back of my head. Mark tells me no one can see them because of my hair. Phew!

Unanticipated events happen all the time. The bigger and less cross-functional your project, the less impact they have. But that’s because you’re barely making progress as is. The smaller the project, the more cross-functional the team, and the more people depend on each other, the more impact these events have.

It’s not possible to plan for these events, except to have some contingency or slack in the schedule.

For several years, when I was scheduling projects, I would assume we would lose time to weather events in the winter and power outages in the summer. I didn’t know when they would happen. I was sure they would. Those are anticipated events, even if they’re unscheduled.

But unanticipated events can be happy, such as a key person getting married on the spur of the moment. (We had a quick party, he left and returned a week later after a honeymoon.) Or, they can be injuries, such as mine. To be honest, an afternoon of lost work is not the big a deal in the scheme of things.

So when you schedule, remember those unanticipated events. Give yourself enough of a contingency plan to deal with them. If you’re working in an agile lifecycle, your velocity will self-correct. But don’t assume they’re not going to happen. Something will. You just can’t tell what.

It’s been quite the Monday so far. My office toilet started spewing water, a cabinet door fell off one of the cabinets in the kitchen, and I’m trying to back up and duplicate my hard disk because both latches on my Powerbook broke at the Agile conference and I need to send my computer off to be fixed. And of course, I have deadlines for presentations and articles, and the PM book I’m trying to write.

I have a theory about this string of events. I just came off a crazy amount of travel: 9 out of the last 12 weeks, I’ve been out of town. I realized about halfway through that was too many weeks. Oh well. But the problem with that much travel is that I don’t use things (like the toilet or light bulbs) in my office. Entropy happens.

Entropy happens on projects too, especially if we don’t pay attention to pieces of the project or use certain tools/processes infrequently. One of my clients didn’t realize they’d broken the build in a way that required three weeks to fix until after six weeks had passed, because they only built once every couple of months. One client didn’t realize they could no longer generate the documentation system until they had to produce a one-off for a quick fix for a customer. Producing the documentation took them longer than the emergency patch.

Short iterations help. If you’re starting a project, a Hudson Bay Start works. On a project, just asking the questions about the infrastructure can help people see if they should try something.

In the meantime, I have more deliverables before I can ready my computer to go away for a week or two. While I’m getting the computer ready, I’ll be looking at the other kinds of infrastructure risk in my office, to see what I need to fix.

My friend and colleague, Stever Robbins, has started a blog, and one of his early posts is Are engineers living on another planet? Don’t they use their software?

Unfortunately, not always. It takes self-discipline and the desire to look for problems to cause people to create systems that allow them to use their own software. If a project team only builds once a week, they’re not going to use their software. If they fix a bunch of defects at one time, the testers can’t do a complete install and test pieces in isolation. Instead, the testers need to install the whole darn thing and test everything together.

The current phrase for using your own software under development is “eating your own dog food.” (Anyone know the origin of that phrase? I’m fairly sure I was using in the 80’s, before Microsoft popularized it.) It’s not easy to use the product under development. And, it’s a great idea.

I changed ISPs last Friday. At some point Friday, my ISP bounced my email with a strange (to me) message. This is the same ISP that had problems just a few months ago, so I was done. I need email up virtually 100% of the time. And if I can’t receive email, I need my ISP to collect it, not lose or return it before delivery to me.

I’m not privy to how my old ISP tested changes to the mail queue. But I can imagine the conversations people had about the changes and how they might test them. If they were anything like some of the systems I’ve worked on in the last 25+ years, there were comments like this:

• We’ll wait till the system starts to degrade and then we’ll intervene manually.
• We’ve tried simulating the system, and we just can’t tell enough. We need to put it in place and respond to the degradation.
• We’ve measured and monitored the system; we won’t have problems.
• We’ve architected the system to degrade gracefully. We’ll watch for that and respond.

I have yet to see a system degrade gracefully. I’ve seen systems show small warning signs before degradation–signs that were sometimes too small to notice as the first instance of degradation. But we, the human beings who were supposed to respond to those signs, more often than not completely missed them. It was too easy to talk ourselves into thinking the warning signs weren’t really signs. (BTW, this works for the human body, too.)

If you’d like to know how your system degrades and when, you gotta test it in a variety of ways: stress, load, performance, reliability testing. This is not easy and requires significant knowledge of the internals of the system under test.

I don’t know what kinds of testing my old ISP did. But my new ISP seems to have a lot more redundancy and testing of changes before making the changes. And, they don’t have to make the changes to everyone all at once–a nice risk management technique.

If you’re concerned about system degradation, take the time to think about how you’ll test it, especially if you’re selling a commodity product where your customers have options. Otherwise, your customers may well do what I did–walk.

One of my students emailed me recently, asking about how to assess project complexity. He said, “I think it would be pretty neat and also quite useful if you could define a project as say a .60 Apollos or what have you… I don’t imagine it would be at all easy to come up with the measurement though…”

It would be cool. And, I think complexity is relative to each organization–especially its project culture and the people. Remember the Project Pyramid? The parts that make complexity difficult to assess are in the two areas hardest to measure: People and the abilities, and work environment.

So when I think about project complexity, I think about the complexity in this environment. And I think about what these people can do. Those two risks are impossible to measure, and have a huge impact on perceived project complexity. Add in short schedules, large feature sets, and a low tolerance for defects, and you’ve got an even more “complex” project.

I’ve had good results talking about project risks rather than project complexity. That way I can explain the whole picture, rather than lump everything together as “complex.” And, maybe we can make some tradeoffs

I’ve been watching, reading, and listening to the Katrina coverage over the past week. And the one thing that stands out for me is my perception that there was a lack of disaster planning.

I’m not going to play the blame game–there’s plenty to go around. But here are the questions I would have assumed the managers in charge of planning would have asked:

• What’s likely occurrence? After a hurricane, we can plan on no electricity, which means impaired communications, no air conditioning, people needing to use generators, and at some point, a lack of water. How long is this likely occurrence going to happen?
• What’s an unlikely, but not out-of-the-realm-of-possibility occurrence? This is where I’d assume the electricity would be out for several days, maybe a week. Remember, all of our systems need electricity to continue to run.
• What would be a disastrous occurrence? This is the flooding scenario, where pumping out the water is impossible until after the levees are fixed.

The value a manager brings to a project or to an organization is planning the work and preparing for risks. Risk management is not a one-time planning event, but starts with planning (risk discovery) and continues as the project and/or the organization continues.

The one conclusion I’m drawing is that too few people performed risk discovery and developed mitigation plans. Don’t let that happen to your project or organization. You don’t have to be paralyzed by risks, but the more aware you are of the potential risks and the more you plan for dealing with those risks, the better off your project or organization will be.

I asked my project management students to create a project dashboard for the projects they’re organizing, so we can talk about what happens when the indicators go up down or sideways. One of the teams came up with a “risk constellation” chart — which I thought was a brilliant way to show risks. (If you’re wondering about the probability axis, I made data whole numbers instead of percentages, because I thought the image would be easier to read.)

I love the fact that now you can see how risks cluster — or not. I’m filing this under the category of “I wish I’d thought of it.”

At Boston SPIN last night, Tim Lister of “Waltzing with Bears” fame gave a talk about recognizing and managing risk. It was great. If you ever have a chance to see Tim speak in person, do so (Yes, Tom DeMarco is also an excellent speaker, but he wasn’t there last night :-).

When I manage risk in a project, I ask myself “What’s the worst thing that could happen?” For the schedule, it’s the choice of lifecycle and practices that could help or hurt the schedule. For the planning it’s not knowing enough and building a bad plan. For the people, it’s hiring or assigning people who can’t start on time, or don’t know how to work on the product.

What I see most often when I perform assessments is that the people don’t know how to develop or test the product. There is so much learning the project staff requires that the time estimate is off — sometimes by orders of magnitude. So when I manage a project, I think about all the other risks, but the one that usually hits the top of my list is: “How can I manage the risk that the project staff will be unable to develop the product?”

Here are some ways I’ve managed that risk in the past:

• Ask the project staff what they can show the entire project team early: designs, models, simulations, early runs of just the algorithms, early plans, etc. Once they show something at one stage, I ask the question about the next stage. (First I might see a high level design. Then I could see an algorithm. Then a prototype of just the algorithm. That’s what I mean about the stages.)
• Ask people to show me visible progress. Telling me they’ve completed something is not the same as showing me output or someone else’s review of a work product or performance measurements. Visible progress is necessary.
• Ask the project team to view the work as a problem solving exercise. What three alternatives (minimum) have they considered? What three things can go wrong with each alternative? If they can accomplish the design, is there some other problem I don’t know about — so we can go about solving those problems?

If you’re working on a risky project — probably the only projects worth doing — then a huge risk is not knowing whether you can accomplish the work. Apply the ideas of small deliverables, visible progress, and problem-solving, and you’ll manage the risk. You may not eliminate the risk, but you’ll manage it. As Tim said last night, “Bad things might still happen. But disasters won’t happen.”