Managing Product Development

I’m writing the portfolio management book, and I just finished a whole big re-architecture. I’m so excited.

I realize most people aren’t that excited about a rearchitecture :-), especially not of a book in progress. But I am, because I took my own advice.

When I started writing the book, I had several partly done chapter-things. They were not particularly well-written, nor were they coherent and several pieces were tightly coupled. But they were enough for the Prags to see what I was thinking. Luckily, that was enough for a contract.

I’ve been writing off and on since I got the contract, and have been getting stuck. I realized last week it was time to print the book and start cutting pieces of it to reorganize.

I finally started making the book (yes, I write in markup language, check my writing into Subversion, and use make to make the book), and seeing it on paper helped me see where my features were.

I have some user stories:

• “As a first level manager or technical lead, I want to see how to make a portfolio.”
• “As a middle manager, I want to see how to make a portfolio and make decisions about it.”
• “As a senior manager, I want to review the portfolio, and make decisions about it based on data.”

But being your own product owner is not such a good idea. Because I thought the roles were driving the book, I had separated a bunch of the writing by role first, and then what the roles did. But it turns out, that for this book, right now, the portfolio activities are what needs to drive the book. Maybe that’s obvious to you. But it wasn’t for me.

I realize the current book’s architecture may not last. But I can see how to write more of it. And, I’ve been refactoring to clean up my writing. I think of the refactoring as where I put things to make the book clearer, and editing as how I change the words to make the ideas clearer.

I wrote several features–actually parts of several features because I got stuck. Now I’ve rearchitected and the writing is flowing. I’m probably not done rearchitecting, but that’s ok. I have a place to head towards now. Onward!

I’ve taught several workshops where people wanted to learn how to start adopting some agile approaches. They knew about timeboxing, but didn’t quite see how to make it work. The part they were missing was having working valuable product at the end of each timebox.

I explain that to the participants, and they nod sagely. Then I ask them to do a simulation. Practicing in a workshop is a safe place to practice. If they don’t do something quite right it’s still ok. Normally this part proceeds well. Some folks have a little trouble, but most people start changing how they work by the third iteration.

At a recent workshop, things didn’t go so well. One group took five iterations to complete the first part of the project when I’d expected three iterations (when designing the simulation). But take a look at a chart of their first project’s velocity.

They couldn’t predict what they could finish–or not–in an iteration. One of the reasons they had so much trouble is that they did not implement by feature. They attempted to build the architecture first. But as soon as they started attempting features, the architecture didn’t quite work. Instead of throwing away a not-quite-right architecture at the end of iteration 2, they decided to stick with it, because of all the time they’d invested.

They had a different experience in the second project. In their first iteration, they decided to do an architectural prototype, but not deliver anything. Ok, I could live with that. They had a working architecture, and if they’d put things together, they could have been halfway done with the project. They chose to leave the first iteration as a prototype.

In planning for the second iteration, they told me (senior management), they were going to work on the prototype more. I asked what they would have to show me. It wasn’t going to be product, so I told them that was unacceptable. They didn’t plan much for that iteration, but delivered plenty. Same thing with the next two iterations.

They never did have a predictable velocity, because they were caught in the trap of pre-planned architecture instead of seeing the architecture evolve as the product grew. I asked them if this ever happened in their organizations. Predictably, it did.

This may be the most difficult idea to help people see–that they make more progress implementing the most valuable features one at a time, and letting the architecture evolve. Big Design Up Front (BDUF) slows down the project. You can’t predict what the architecture needs to be.

Labels: architecture, BDUF, implement by feature, timebox

Udi has a great post, Money?! Schedule?! But I

Thierry poses a question I’ve heard in several of my PM workshops this week and last week: When should the team do the architectural work?

Thierry’s concerned if his team continues to implement by feature, how can the team do the architectural work? If they take an iteration or two to deal with architecture, they stop churning out features. And, Thierry has an additional problem–his new module could be a stand-alone feature. But he has some other alternatives:

• Implement three features and see where the architecture is going. If the team implements three relatively independent features, it may be clearer where the architecture is evolving. Once you see the evolution, maybe you can spend a short timebox (couple of days?) to define the architecture big picture, and let it continue to evolve within the framework the team has defined.
• If trying some features doesn’t work, prototyping several architectures might. When you prototype, you don’t have to have releasable software; you just need enough to evaluate the architecture. If you prototype, maybe you don’t have to take the whole team to do so, and they can still be making progress.
• Develop tests for the architecture. Get enough of the team in one room, have someone facilitate the discussion, and generate the questions/assessments/tests you’ll need to use to know whether an architecture will work.
• Timebox the architecture development. Spend a day or two (I would definitely not spend more than a week) developing an architecture on paper, and then start implementing some features so I could see if I got it right.

I’m sure there are more options and I can’t think of them now.

If you do take time to do some architectural work, make sure it’s as little design up front, so you can test the architecture by implementing more features. Any time you do incremental development, you do run the risk of not realizing the architecture is sufficient until later in the project. But that risk is mitigated by having significant parts of the product already working.

Labels: architecture, incremental development

Joshua asked me to clarify what I meant by implementing by architecture. Here’s my picture-story.

When a team implements by architecture, they tend to be functionally-based teams implementing across the architecture. See .

When a team implements by feature, they are cross-functional teams. See .

When teams implement by feature, they do what’s needed in whatever part of the architecture is needed. To the outsider, it looks a little messy–sometimes incomplete. But to the team, the architecture is cohesive. And, most importantly, the team is not writing code they won’t use.

When teams implement by architecture, they are too likely to implement more than they need–ever.

Given the thoughtful comments on Architects Must Write Code and Testing Design, I’m wondering if some of the difference in our beliefs stem from our perceptions of the architect’s role.

I see the architect as the technical lead who shepherds a product through the overall design, someone who explains enough about the system and how it hangs together so that the other developers can take their parts write/design them (in whatever way works for them). I now believe in as-little-as-possible design-up-front and as-much-as-possible emergent design. Because I see the architect role as a shepherding role, I see the architect as immersed in the project, and integral part of the project team who needs to continue to participate to see when things go south.

But I wonder if some of you are thinking about the architect as more of a consultant. The more up-front design and architecture you do, the more you can try to create the architect-as-consultant role. Certainly, architects for buildings do become “merely” consultants once the general contractor starts the actual construction. (Yes, this is one more reason the building construction metaphor doesn’t work for me.) If the role of the architect on your project is more of a consultant, then I can understand why you disagree.

I’m going to have to think about ways to make the architect-as-consultant model successful, even if I don’t believe in it. (For me, this all points to feedback for the architect.) Certainly, many of my clients believe in and use that model. Just because I can point out ways that it doesn’t work doesn’t mean these folks can or are willing to change. So, I’ll be thinking about ways to make it work–and the risks involved. (This will make the organize-your-project-team chapter easier to write too. Duh.) I may well ask you for your comments.

In Architects Must Write Code, several architects responded that I was too prescriptive (I’m summarizing their comments). Maybe. But I don’t think so.

I’m in a nice hotel, where things just don’t work completely right. Yes, the hotel is clean (that’s the big thing with me). The hotel upgraded me to a suite with an oval bathtub. Clearly, people of normal height and weight had not tried to shower here–the tub is about 9 inches (a hand-width) from the toilet. There’s no grab bar to balance on one foot while stepping into or out of the tub/shower. I tried to draw you a picture here.

I travel a lot and stay in lots of hotels. This problem is similar to many problems I encounter in hotels: outlets too far away from where I want to use them, chairs too high, sinks too high, and my favorite, too-high shower heads. Most of the time, the shower head is too far away for me to adjust. I’m short (5 feet tall), but if I can’t reach the shower head, it’s set for a 6-foot plus person. If the people who designed the hotel rooms had tried them at all, they would realize they were creating difficult situations for a significant percentage of the users.

So, it’s possible that architects don’t need to code and participate in a project. But I don’t know of other effective techniques to test the design. Testing design with a thought experiment is insufficient. Testing design by using it provides much more feedback.

However you arrange your project, think about how to test the design–the design in-the-large, and all the little pieces of design-in-the-small. Certainly, test-driven development is great for testing design-in-the-small. And if you have a whole project that’s willing to try test-driven design for overall architecture, that’s fabulous. But if you don’t or you can’t see how, at least think about how to build in testing of the design, especially design-in-the-large as you proceed through the project. Then you won’t end up with a bathtub practically in the toilet, requiring people to balance while stepping in and out of the shower.

I had the opportunity to read Practices of an Agile Developer: Working in the Real World. The book has 45 tip to help developers become agile. And, it’s clear that Venkat and Andy know the problems of becoming an agile developer, because along with each tip, there’s a devil-thought to show people what happens in the real world. There are also angel-thoughts to show people why this tip works.

My favorite tip, and something I’ve been saying in my assessment reports for the last 10 years is “Architects Must Write Code.” (p. 155 in this book.) Venkat and Andy say this about ineffective architects:

They typically come in during the beginning of a project, draw all kinds of diagrams, and leave before any serious implementation takes place. There are many “Powerpoint architects” out there, and they aren’t effective because of lack of feedback.

I’ve been a part of projects for 30 years. I’ve been assessing projects for 10 years. And every time I’ve seen an architect who doesn’t participate in the code-writing part of the project, I’ve seen an architecture that doesn’t do what it’s supposed to do, never mind extend to inevitable changes in requirements that occur during the project.

Architects need feedback about their architecture. The only way to get feedback is to write the code, integrate it, and see what happens.

If you haven’t read Joel Spolsky’s entry on office architecture stop and read that first. Finally, an office in which people can successfully work alone andwith other people — and who don’t have to worry about keeping their voices down. I’m amazed at the space per person (425 sq. ft if I understood). Most offices give people 80-100 sq. ft. My home office is 22×14, 308 sq. ft. I have room to spread out my various current projects, room to meet with one or two people, room for a flip chart and of course all the machinery I need. The offices must feel palatial to the people at Fog Creek.

What Joel describes is requirements starting with the users. Notice that he doesn’t separate functional requirements from non-functional requirements. What people call non-functional requirements are the attributes of the system that make it usable — or not. When you start with the users, and describe requirements holistically as Joel did, (well, ok, he has some design, such as “offices with doors”), you describe what makes the system (in this case, the offices) usable.

Esther and I are teaching consulting skills to architects this week. We have a variety of simulations as part of the workshop, and we’ve noticed that great architects act in certain ways:

• Great architects look for multiple alternatives to explore.
• Great architects look for how the user claims to want to use the system, and continue to probe on currently-visible extensions for how people may want to use the system in the future.
• Great architects discover the primary system attributes that will drive the system and build on that. One architect described this as building from the inside out.

The primary system attribute, what other people call a non-functional requirement, is often unclear. For our simulations, the attribute was the ability to support a physical load. For software systems, it can be some aspect of performance, reliability, security — or some combination. Here’s an example. Our telephone systems in the USA are build around a primary system attribute: the system will have uptime of greater than 99.999% over the course of a year. (The telco requirements are probably higher than that, but it’s a lot of 9s after the decimal.) That means that you can virtually always pick up a land-line phone and get a dial tone. That requirement drove the architecture of the telephone systems, both for each central office and all the interconnections.

If you find that primary attribute, or the combination of attributes, the system architecture alternatives appear — to me it looks as if they become clear to architects by magic.

The lesson I’m taking away this week is that if you want a coherent architecture, it’s more important to discover the primary system attributes rather than all the functional requirements. No matter how you approach requirements, make sure you take a holistic approach. Don’t use techniques that separate functional and non-functional requirements. If you do, you won’t see the architecture emerge.