Managing Product Development

It’s winter here in Massachusetts, and we’ve had lots of snow, ice, rain, snow, ice, snow, ice, rain. All that freezing and melting plays havoc with the roads. We have lots of potholes, and the local and state governments are busy doing emergency repairs all over the place. (For those of you who don’t know how potholes are made, first the water seeps in through the cracks in the roads. When the temperature drops and the water under the road freezes, the cracks get bigger. Loop until the road surface breaks apart and creates a hole. When the holes are deeper than a few inches and longer than a few inches, it’s time for emergency repairs. We have some foot-long and foot-deep potholes right now. Yuck.)

Potholes are a fact of life in the northern climates when you have cycles of freeze and thaw. But I’ve been talking to some folks whose defects are like potholes. “We only have time for emergency fixes.” “We have time to fix and fix the same darn thing over again, but we don’t have time to do it right.” That’s when your defects are like potholes.

Potholes slow down traffic, because the cars need to drive slowly through them, or around them. Defects, especially big ones, slow down other development or fixes. So, what do you do?

If you have a ton of defects, I would choose a one-week timebox, and work on fixing them. For me, fixing means developing a fix along with a unit test (or two or three), getting some peer review, and then checking it in so the developer can do some around-the-area testing before system test. I don’t care if the developers write the unit test first, I just care that they write some unit tests. Although, if you’ve got defects, you’ve got the makings of a bunch of great unit tests. I would not allow any development in this timebox, just fixing and checking the fixes in a variety of ways.

After making some progress and getting the defect counts lower, decide if you want another one-week timebox. Is it more valuable to finish (fix defects) than write new stuff and have old stuff unfinished? I don’t know, but you do for your project.

For the rest of the project, I would make sure people are working by feature, developing and fixing and checking a whole feature in totality before moving on to the next one.

If you’ve got a bazillion defects, they are like potholes–they prevent you from moving smoothly onto the next piece in your project. Fix them! No emergency repairs either–real fixes.

It’s been a heck of a week. My office is in my basement (a walk-out basement with lots of light–it doesn’t feel like a basement). Earlier this week, I thought I had a leak in the foundation–there was a small damp spot in the rug.

I called the basement people to make appointments. Of course, the appointments were for over a week away. My office is now beginning to smell of mold. Ugh. Luckily, one of the people told me to pull up the rug so I could see where the water was coming from.

Mark figured out how to pull up the rug (between the baseboard heaters and the carpet being tacked down and the pad glued down underneath, this was not a trivial matter). Now we can see the wet spot, which is larger than I imagined. But, pulling up the rug helps the floor dry out (as well as the rug and the pad).

A couple of days later, I realize the water is not coming from the middle of the floor as I’d originally thought, but was coming from a wall, near one of the baseboards. Another day later, we realize the water is coming from a hole in the pipe that feeds the baseboard (forced hot water baseboard heat).

Oh. Now I know what to do. This is the kind of leak I know how to fix. (Not me personally, but I know who to call.) The heating guys fixed the pipe yesterday, and the insurance company’s fixit-guys disinfected the carpet and installed the dehumidifier to eliminate the water in the rug and the wall.

I still can’t work in my office; it’s too noisy and smelly (but good smelly from the disinfectant). Monday, they’ll come shampoo my carpet, and then all we’ll need is insulation back in the wall where the baseboard is, and my office will be fixed.

The point of this long story is that defects can take a while to become obvious and the symptoms don’t necessarily point to the actual cause of the problem. We finished that part of the basement 14 years ago. We hadn’t had a problem with the heater there, and I’ve been in that office for almost 12 years. But the joint in the pipe just wasn’t strong enough and after heating the room for 14 years, gave way.

The fact that I saw the water in a place that wasn’t the source of the leak wasn’t a surprise, but again, mimics what happens when we find defects in software systems. Defects cause problems in lots of places and determining the source of those defects is harder the longer the defect has been in existence and the longer it’s been active.

So, I’m in the kitchen, not in my comfy chair with my phone at my side. I may move to the couch later, because these chairs are not made for sitting in for hours at a time. I’m not as productive as I normally am–but my office will be fixed and habitable soon.

The lesson for me is: as soon as you detect a defect, start looking for the cause. If I’d noticed the wet spot in the floor the day it occurred, I might have been able to trace it back to the pipe. (Probably not, the rug masked the origin of the leak. But maybe.) So in order to start looking for the cause, I want to make my defects as obvious as possible as early as possible. Easier said than done, but in software, it’s much more possible than with houses.

Well, all the comments on my No More Bugs post got me thinking. To answer a few comments here and elsewhere,

• I’m not trying to blame developers for being human and making mistakes. When I was a developer, most of my mistakes involved code. As a manager, most of my mistakes were in my dealings with people. Fixing code is much easier than fixing relationships.
• I’m not trying to play word games. To me, the intention of bug vs. defect is different.
• I’m willing to admit I’m alone in this thinking :-), or at least, I don’t have much company (yet).

My intention with naming mistakes as defects is something I said in an earlier article:

It wasn’t just that I wanted the project team to be aware of defects, I wanted them to be comfortable with acknowledging defects. They had to realize that they had produced defects and that they would not necessarily be able to fix all of them for a given release. That, in turn, makes awareness of defects less overwhelming. It makes the defects bearable.

I hope this post and the other one has started you thinking about how we create problems and we prevent them. Your comments have helped me think more. Thank you.

During an email conversation last week, I suggested that my client change his naming of “bug” to “defect.” He asked why.

People don’t create bugs, but people create defects (or problems or faults). Bugs crawl in or fly in from the outside, when you’re not expecting them to. But we expect defects and problem in software. We can take actions (test-driven development, pairing, peer reviews, inspections, and more) to reduce the number of defects in our code, but those actions are very different than preventing bugs from flying into the house (putting on screens, closing doors). And, if we’re outside our houses, we can’t take similar actions to reduce the total bug population.

It’s possible that the last real bug was the first one, the moth caught in the hardware. Maybe there were more physical bugs that caused hardware to short out. But no longer. We rarely, if ever, have physical bugs to worry about.

But we have plenty of defects. I am convinced one of the reasons we have so many defects is that we call them bugs. When we call problems bugs, we avoid taking responsibility for the problems (defects or faults) that we have caused. The bugs didn’t just fly in and land on the hard drive. As the developers create the code, they also create the defects.

The first step in solving problems is to acknowledge that you have them. If you want to deal with the defect problem, stop calling those problems bugs. Bugs may be an acceptable name, but it’s not a helpful name. Call the problems defects or problems or faults — something that will help people realize they need to examine why it’s acceptable to allow defects into the code.

Call a spade a spade. And call those problems defects. They’re not bugs; they’re defects. Then maybe you can determine how to get rid of the ones you have and not make more.

I was sitting on the couch was organizing a database last weekend, with daughter #2 sitting next to me. I was creating a script to go through each record removing a field’s contents and adding new contents to another field. Not a difficult thing to do. I created a loop, and daughter #2 said, “Hey, how does the computer know how that loop will end?” She’s not a developer. I don’t think she’s seen the inside of a program. But she knew enough to see that I had an infinite loop.

I’m a pro at creating infinite loops. When I was developing, I would regale Mark with my infinite loops. But when I was a developer, I had a notebook in which I marked all my normal techniques for writing infinite loops. I don’t write many scripts these days, so I don’t track my loop tendencies. Which is why it was even more important for someone to be sitting next to me while I was working.

I took advantage of the situation, and asked her what I should do instead. She pointed out what I could do, I suggested another technique, she replied with another suggestion, and we both jointly agreed on what to do. The whole conversation took maybe 5 minutes.

The cost of the infinite loop was small, but instead of having to clean something up, I was able to write it correctly and continue with my work. It would have taken much more than 5 minutes to clean up that mistake.

Pairing while I work has been cheaper for me, in terms of preventing defects. Peer review after I write something is next, followed by formal inspection, followed by me finding my defects and fixing them. I was particularly pleased with this pair experience. Even though I didn’t impress daughter #2 with my software development expertise Better to be unimpressive than to spend more time fixing.

I’m familiar with managing defects by severity (how bad the problem is for the user if the user encounters the problem), and by priority (what’s the business value of fixing this problem), but I had lunch yesterday with some folks who use frequency of occurrence to also manage defects.

They started this because they have a huge customer base, and if enough people perceive there’s a problem with the software, there is a problem. And, they include downstream customers (i.e. people in the building who may be customers even if they don’t pay for the software) as part of the users.

Here are their definitions:

1 - High: Encountered by many users, including downstream teams, in their normal course of work (> 10% of the user community or > 100 individual users)

2 - Medium: Encountered by some users, including downstream teams, in their normal course of work

3 - Low: Encountered by few users, including downstream teams, in their normal course of work (< 1% of the user community and < 10 individual users)

These weren’t easy definitions to settle on. Their product has substantial internal computation and a substantial GUI component. These definitions appear to “punish” the teams responsible for the GUI, while the internal computation teams seem to have more flexibility. (But what they’ve realized is they need to modify the way they define and implement the GUI. The piece I particularly like is that the product architecture folks have everyone (the rest of the developers and testers as wel as customers) as part of the customer base.

I don’t think this works for everyone, but I like the idea of saying, “Who’s affected by this problem? If there are lots of them, let’s deal with it.”

I have several accounts with a credit card company: two cards and a merchant account. They don’t want the expense of printing monthly statements for the merchant account, so they sent me a letter to enroll my merchant account in online statements to avoid the paper charge.

In my default browser (Safari), I attempt to log in. I can’t (Problem #1). In fact, I end up in browser limbo, with some message about too many redirect (Problem #2). I try to log in in Explorer. I can’t. It tells the me password is wrong (Problem #3). I call the 800 number three times before I get someone who tells me I need a new user id. I ask why. The nice lady says, “We keep our businesses separate, so you need a user id.” I say, “But (and I’m starting to rave here), the web UI looks the same. I have tabs at the top of each page that allow me to move back and forth. If you’ve integrated the web page, why not integrate the user ids???” (I’m definitly speaking loudly, because I’m so angry with the stupidity, waste of my time, and lack of documentation.) Poor lady just keeps saying they keep their business separate (Problem #4).

These are serious problems. I’m not a UI designer — I’m a user. This makes me an expert — in using computer systems, not designing them. The problems:

1. If a user attempts to log in and can’t, catch the error. Safari shouldn’t have tried to deal with this error, the web site app should have. Catching an unable to log in error is basic programming. How could they have missed this?
2. Why do you care what browser your users use? The world is full of browsers. That’s why we have Java. Accomodate all the browsers. Sure it means that you have to write code more carefully and test on many platforms. Is it worth losing any customers because your developers were too lazy to write good code?
3. Make sure the error text matches the error. There was nothing wrong with my password; they wanted a new user id. The problem is the text on the page. The text says “New user? Create an id.” But I’m not a new user. I manage my credit cards online every month. I’m an experienced user. The instructions were inadequate, as well as the error text.
4. If you have a consistent look and feel to the site, and you’re trying to seel multiple services to one customer, don’t make the customer have multiple ids, one for each service. I’m sure that having multiple ids makes it easy for this company to track revenue by division. But it makes it much harder for me to manage my user ids and passwords. Who is the customer here?

To add insult to injury, I can’t write this in an email and send them feedback. (Problem #5) I can call (and spend more time on hold? No thank you), but I can’t write. So, I’ve now made a public example of a stupid website. Great. When I make mistakes, I want people to tell me, not write about it somewhere public. Gotta wonder why these people don’t care.

Ok, tirade about stupid UIs off. Back to work.

I’ve had several discussions about maintenance in the past few days. I’m beginning to think I have a different definition of maintenance than other people do :-). For me, maintenance is fixing problems in code. Maintenance is short, small, well-contained and code-based, and should be fixed by the developer(s) who created the problem.

So what happens if there’s a requirements problem that led to a design problem that led to a larger (greater than a couple of days worth of work) development problem? To me, that’s a short project. Once the problem is larger than just coding errors, it’s not maintenance, in the sense that I think about it. If you have to rewrite requirements or fix the design or involve more than just the one or two authors, it’s bigger than “maintenance”. And once it’s a small project, I can manage it with all the other small projects that are part of a release.

This means that when I work on projects, I see the work differently than others seem to. Because I believe everyone has to fix the problems they created, and because I see maintenance as just fixing those problems, I don’t have a problem with maintenance. I do have a problem with scope creep, but not in the sense of new requirements. I have a problem with trying to finish the work the project staff said was already done but isn’t.

That’s why I ask people to plan inch-pebbles, why I use milestone criteria, why I ask developers to implement by slice instead of by architecture, why I request that technical staff completely finish one thing before they go on to another. A by-product is that I don’t normally have to deal with maintenance, unless I’m coming into a project after it’s already started.

What’s your definition of maintenance?

In the last few weeks, several people have asked me how much rework is normal. Well, if you’re working in a test-driven development environment, you probably have very little rework. My estimates for the few real test-driven projects I’ve seen is that they spend about 10-15% of their time on rework (finding problems and fixing them after the technical staff thought they were done with the particular feature. (This percentage may be this high because the technical staff were learning how to work in an agile way.) They spend lots of time on iterating through the requirements with the customer, which I don’t count as rework.

But in the last few assessments I’ve done, and from other recent emails, my experience is that rework accounts for somewhere between 75% and 80% of a “normal” project’s time. That’s a huge drain from the project. The rework impedes forward progress. I know of these ways to reduce rework:

• See how little you can do, and deliver that much as quickly as possible. The technique I use most often is to break the pieces into groups of requirements/features and then perform iterations within whatever lifecycle the people are used to.
• Use peer reviews, walkthroughs, inspections, anything that forces more than one pair of eyes on the code.
• Pair the testers with developers, or somehow get the testers to test the developers’ code as soon as the product is built. Early feedback from testers can help developers prevent inserting more defects into the code.

All these techniques help developers focus on some small amount of work, and look for problems early in that work.

I wish I could say there was some reasonable amount of rework, but I don’t know how to define that number. Any rework is a problem that you could have caught before. The question you manage is: how much rework should you prevent, and how much rework (time wasted) is ok? (For research-y projects, the answer is lots of rework is ok. For get-the-product-out-the-door projects, any rework is not ok. ) You’re the only one who can answer that question for your project.

Bloglet subscribers saw two posts from me Friday. They saw the post I published *and* the post I saved as a draft. Surprised me.

Since I know about this feature, I’ll work around it, and compose future drafts somewhere else. This isn’t a big-deal problem for me. But it was a surprise, and a defect for me.

On the other hand, if I was part of a project team, and wanted to save some drafts of things I was thinking, and only my team had subscribed via bloglet, then maybe sending around ideas early would be helpful for the team and the project.

Just goes to show you that you have to think about each set of users, either as personas a la Cooper (”The Inmates are Running the Asylum”) or as Gause and Weinberg describe friendly, unfriendly, and ignore users (”Exploring Requirements, Quality Before Design”). Just thinking about the Users isn’t good enough. We don’t all fit the same mold. One user’s defect is another user’s features.

Actively work to define who all the users are, and take some time to describe them, even if you don’t want to use full personas. You’ll have fewer unintentional defects.