The Core Question about Building Better Software

I have never met anyone who could honestly say “I am building software today as well as software could ever be built.” Individuals, teams, and organizations should all be continually looking for better ways to do our work.

Frustration with disappointing results is a powerful motivation for trying something different. But you need to be confident that any new strategy you adopt has a good chance of success. Organizations often turn to the latest buzzword methodology, the hot new thing in software development, as the magic elixir for their software problems. Sometimes, a manager reads about a promising (but possibly overhyped) new methodology in a book or blog and insists his organization adopt it immediately. This is sometimes called “management by BusinessWeek.” Or maybe a developer hears a conference presentation about an intriguing new way of working and wants his team to try it. The drive to improve is laudable, but you need to target it at the right problem.

In recent years, agile software development has been the classic example of this pursuit of magic solutions, so I’ll use that as an example here. Over the years, though, people have leapt onto the bandwagons of numerous new software approaches. They all have merits, they all have limitations, and they all need to be applied to appropriate problems.

In all of these cases, well-intentioned people want their organizations to achieve better performance. “Better performance” could mean many things. Maybe you want quicker delivery of useful products to the customer, or fewer defects in delivered software that annoy customers and drain the team’s time with rework. Many people are drawn to agile because they seek closer alignment between their completed products and customer needs, and they want to respond more nimbly to changing requirements during the project. This is the essence of process improvement: identifying problem areas, setting goals, and choosing techniques you believe will solve the problems.

Before you settle on any new development approach, though, take the time to ask the central question about how to build better software: “What’s preventing us from achieving those better results today?” If you want to deliver useful software faster, what’s slowing you down now? If your goal is fewer defects and less rework, then why do your products contain too many bugs today? If you want to respond faster to changing needs, what’s standing in your way?

In other words—again taking agile development just as an illustration—if agile is the answer, what was the question?

I hope I’m wrong, but I suspect that few organizations do this sort of root cause analysis before they latch on to what appears to be a promising solution. Setting improvement goals is a great start, but you must also understand the current obstacles to achieving those goals. You need to treat real causes, not just symptoms. If you don’t understand those real barriers, choosing any new approach at all is just a hopeful shot in the dark.

Perhaps you want to deliver software products that meet the customers’ needs better than you have in the past. You might be tempted to adopt agile because you’ve learned that agile teams typically include a role called the product owner, who’s responsible for ensuring the product achieves the desired outcome. “Perfect!” you think. “The product owner will make sure we build the right thing. Happy customers are guaranteed.” Problem solved, right? Maybe, but not necessarily.

Suppose, on the other hand, you have that same goal of better meeting customer needs (who doesn’t?), but before making any changes, your team does some root cause analysis to understand why your products don’t instantly thrill your customers already. Root cause analysis is a process of thinking backwards, asking “why” several times until you get to issues you can address with thoughtfully selected improvement actions. The first reason that’s suggested might not be directly actionable, nor might it be the ultimate root cause. So addressing that apparent initial cause might not solve the problem. You need to ask “why” another time or two to ensure you’re getting to the tips of the analysis tree.

Figure 1 shows a portion of a fishbone diagram, a convenient way to work through a root cause analysis. It works well with a group of interested stakeholders and a whiteboard. Let’s take a walk through that diagram.

Figure 1. Partial root cause analysis shown as a fishbone diagram.

Your goal is to better meet customer needs with the initial release of your software products. Alternatively, you could phrase it as a problem statement: “Initial product release does not meet customer needs.” Write that goal (or problem) on the longest horizontal line in the fishbone diagram.

Next, ask your group of stakeholders, “Why are we not already meeting our customer needs?” Now the analysis begins. The list of possible contributors to “failing to meet real customer needs” is long. One possible answer is that the team doesn’t feel they get adequate input to the requirements from end users, a very common problem. Write that cause on a diagonal line coming off the goal statement line.

Again, you ask why not? One member of the group says, “We’ve tried to talk to real users, but their managers say they’re too busy to work with the software team. We’re already supposed to know what they need.” Someone else suggests that the surrogate customer representatives who do work with the team don’t do a good job of understanding and presenting the real needs of the ultimate users. Those second-level causes go on horizontal lines linked to the parent problem’s diagonal line.

A third participant points out that the developers who attempt to elicit requirements aren’t very skilled at asking customer reps the right questions. Then comes the usual follow-up question: “Why not?” There could be multiple reasons, including a lack of training or interest in requirements on the part of the developers, but it might boil down to the root cause that business analysis is neither a core team skill nor a dedicated team role. Now you’re getting to actionable causes that stand between your current team’s performance and where you all want to be. That cause goes on a new diagonal line attached to its parent cause. Keep up this analysis until the participants agree you have a good understanding of why you aren’t already achieving the results you want. The diagram might get messy.

In future brainstorming sessions team members can explore practical solutions to beat those root causes into submission. And then you are well on your way toward achieving superior performance. You might conclude that adding experienced business analysts to your teams could be more valuable than adopting an agile development approach with a product owner team role. Or maybe the combination of the two will prove to be the secret sauce. You just don’t know until you think it through like this.

The insights from a root cause analysis can point you to better practices that might solve each problem you discover. Without exploring the barriers between you and where you want to be, don’t be surprised if you switch to a different development approach and discover that the problems are still there.

Root cause analysis takes less time than you might fear. It’s a sound investment in focusing your improvement efforts effectively. It’s always better to understand the disease before prescribing a cure.