New Kind of Agile: Powered by The Decision Model

(Important Enhancements to Agile Practices)

Agile, formally introduced in 2001 through the Agile Manifesto, has morphed into many variations and been customized within organizational cultures and projects. After 14 years since its introduction, this article raises an important question.

Despite common aspects of Agile variations, most are missing perhaps the most important new ingredient, The Decision Model (TDM). This article explains why TDM enhances the Agile approach. The article also introduces the integration of TDM with Agile. Based on experience with this approach, the article covers 6 basic questions:

• Briefly, How is Agile Different?
• What is Agile Missing?
• Why Does TDM and Agile Make Sense?
• How Best to Integrate TDM with Agile?
• What About Hesitation and Fear of Agile?
• What has been Real-World Success?

Most readers know that the Agile approach favors interesting differences from past approaches. These differences include less rigidity, reduction of formal documentation, more flexibility for changes, and earlier, more frequent software delivery. The result is continuous evolution of the target software through fixed time boxes, called iterations.

A key differentiator in Agile is that testing occurs in every iteration as each iteration produces a piece of working software. Business users validate and test it, uncovering needed improvements and sometimes even evolving the actual requirements. To manage and encourage such flexibility and frequent changes in requirements, Agile projects include different management methods and techniques than those used on traditional or waterfall projects. These different management methods and techniques include the four ceremonies (the Stand Up, Iteration Planning Meeting, the Retrospective, and the Demo), using a backlog for requirements, fixed time boxes or iterations, and early frequent releases.

Nevertheless, across organizations, Agile practices differ. They range from one extreme of “pure agile” to that of “partly agile.” Partly Agile usually blends Agile aspects with traditional ones – the latter being those used historically and successfully for certain business systems. In fact, many organizations choose not to adopt Agile for large projects. More on this later. For now, regardless of pure or partly Agile, something is missing.

The Decision Model was introduced to the public in 2009 [1] which means it wasn’t in the public domain during the first 8 years of Agile adoption and customization. Based on the success, adoption, productivity, and quality of The Decision Model to date, TDM has much to offer for improving all kinds of projects, including Agile ones.

As a quick refresher, TDM separates decisions (i.e., logic or business rules) from process. Doing so dramatically simplifies the process models and enables greater understanding and maintainability. This means business rules are no longer hard coded and lost to the business. Instead, they are translated into TDM representation in a separately manageable (and automatable) form. Errors in logic are found earlier because TDM, by its principles, is designed for easy analysis. Subsequent changes are easily implemented because of TDM’s simplicity.

As a result, TDM ensures dramatic acceleration and completeness of requirements. It greatly collapses the development cycle and saves significant time and money. This suggests that it can play a key role in Agile projects.

TDM makes sense for at least 4 reasons. First, it moves to the forefront of Agile-delivered software the business logic driving the software. This business logic is the business’s thinking about how the software makes the decisions within it – such as is a healthcare claim payable? is input data of acceptable quality? is a new client eligible for simple or enhanced due diligence? what is a client’s composite risk score? All of these decisions rely on business logic to reach a conclusion. They are the very heart of software. If they are incorrect, not only does the software fail, but the business impact can be undesirable [2].

Without TDM, business logic is the dimension left behind in both Agile and non-Agile software development. Before TDM, there was no recognition for it. There was no thought of it having a universal, business-friendly, and holistic representation. Specifying it, validating it, and testing it remain arduous efforts, despite todays’ full collection of Agile practices.

A second reason TDM makes sense for Agile is that a TDM-compliant decision model can be divided in many ways to fit into iterations. For example, the team can assign a list of business decisions to each iteration. The team can decompose decision models into subsets, each subset assigned to different iterations. Teams can also deliver partial decision models within an iteration, using “stubs.” A stub is a technique indicating a model subset that is outside the scope of a current iteration [3]. A third benefit of TDM for Agile is that IT can deploy decision models to various technologies, often in a straightforward manner, minimizing coding. Further, if target technology changes, TDM-compliant decision models remain the same, only deployment changes.

The fourth reason TDM makes sense for Agile is subtle, but may prove to be the most important one. TDM allows for the true evolution of the business logic itself throughout the iterations. Because the business can refine the logic through iterations, the result is near perfect business logic driving the software. In other words, the business logic itself becomes more mature in the business-sense because of its evolution through the Agile process.

In the past 5-8 years, the improvement of TDM on non-Agile projects has proven to be significant, some say game-changing.

Imagine what it can do for Agile! The case study at the end of this article reveals the positive impact of TDM on an actual project.

The integration of TDM with Agile (or with any other approach) must be done correctly. A universal representation alone is not sufficient. Needed is a disciplined approach and supporting principles by which the decision models are universal in look-and-feel, technology-independent, and comprised of error free logic. In addition, especially for Agile projects, these decision models must be delivered, validated, tested, and deployed quickly and iteratively. In other words, TDM must fit within the flexibility and rapid pace of Agile development and it must deliver Agile benefits.

The good news is that TDM already does all that, just by its very nature.

Below are 3 important enhancements to Agile practices to maximize the success of Agile itself as well as TDM’s role within it. These enhancement address 5 disadvantages commonly cited for Agile. For more information on how these enhancements fit into the overall Agile approach, see link to quickstep)

#1: A Rigorous Framework to Align All Stakeholders

One of the challenges cited for Agile is the impact of scope creep resulting from time-boxed rapid iterations.

To minimize this negative impact and maximize business vision, start with aligning stakeholders using the Zachman Framework for Enterprise Architecture [4] Row 1. Doing so documents the scope of the project for each of the six columns of the framework: the What, How, Where, Who, When and Why of the project. See Figure 1.

Figure 1: Using Row 1 of the Zachman Framework

So, the deliverable consists of the start of a project glossary (“What”), list of processes (“How”), locations of activity (“Where”), potential actors and stakeholders (“Who”), important timelines (“When”), and key objectives (“Why”) for the project.

#2: Set of Suitable Models and Including The Decision Model

Another challenge for Agile is deciding how much modeling is needed.

In general, Agile approaches aim to minimize modeling so that software delivery begins sooner. To determine how much modeling is needed, use Row 2 of the Zachman Framework and select a “minimal” model type for each column. Minimal, in this sense, means focusing on what the business audience needs to validate. Zachman Row 2 models are for business audiences only. They are models that best represent the content of the corresponding column. Use Row 2 to choose the most appropriate model for that column for the target system. See Figure 2. These models are to be developed at a high level, sufficient to ensure an understanding by the team and the stakeholders of the topography of the system, as well as the essential features and requirements. With these models IT team can develop the conceptual architecture [5].

Figure 2: Model Types from Zachman Framework Row 2

To be minimal, for example, process models are developed to a Silver Level 1. [6]

Another challenge is how best to capture business rules and logic, since lists and spreadsheets are time-consuming, error-prone, and complex.

The solution is TDM, which is the new model for Row 2 of the Zachman Framework for the WHY column. These decision models can be developed first to a skeletal level meaning they consist of Decision Model Diagram for which the Rule Family Tables are not yet populated.

TDM is a “minimal” model for Row 2 because there are no technical considerations or artifacts in TDM. Specifically, TDM methodology allows business people to identify information items in business vocabulary (no need for data models, database names etc.) and to populate the decision models using business operators and functions.

Another disadvantage cited about Agile is that testing can be exhausting.

That’s because each iteration involves requirements, automation, and testing (development lifecycle) of its target software. With TDM, non-technical people can apply TDM principles, prior to testing, to find logic errors either manually or with software. This means testing occurs against logic that is already deemed to have no gaps, no redundancies, and no inconsistencies. It is already logically whole. The purpose of testing is whether this error-free logic representation is actually the appropriate logic for the business test cases.

#3: Visualization

Another disadvantage sometimes noted about Agile is that its iterative delivery and flexibility for change means there is less predictability of the final software product.

In other words, a common vision of the final software solution remains elusive such that no one knows with confidence what it looks like or its scope.

Visualization is a technique for eliciting requirements at both high and low levels coupled with a software delivery approach. It simulates the principal user interaction scenarios of the system. The target software is visualized before coding and users can understand the look, feel and behavior of the system before, during and after development. See Figure 3. It can short-cut business vision of the eventual software, early in the project as well as for each iteration.

Figure 3: Visualization Levels [7]

Towards the end of each iteration, the project team provides the Product Owner, SME’s and key Business Team members with a demonstration of what was completed during the iteration. The visibility of how the project is progressing is a key reason for doing these demos. A demo with a larger audience is useful every 3-4 iterations when the release comprises of 8-12 sprints. The demo provides a good forum for obtaining feedback from business users and stakeholders on what they would like the software application to perform in the end state.

The complete set of requirements for each iteration is comprised of user stories, acceptance criteria, models, and visualization. The SMEs and business team validate, or sign-off on it. Then, the business team tests it before it is committed to code, reducing the risk of false features or incorrect requirements. The validation step also ensures alignment of the project team and the business team on the set of requirements for the iteration.

As indicated earlier, many organizations choose not to adopt Agile for large projects. These organizations believe it is not suitable for them. However, the case study below supports the adoption of TDM-Agile for such projects and is testimony to its successful adoption at the enterprise level.

In December of 2012, a leading U.S. financial services corporation needed software to comply with a regulatory compliance mandate. They had tried using Agile before but with decidedly mixed results.

First Steps

After understanding the Agile approach described above, an initial time-boxed project plan was drawn up for release 1. The plan encompassed an initial kick-off, Agile training, an aligning and scoping period, and the 1^st iteration of models. There were several development iterations of several weeks each, and additional weeks for User Acceptance Testing (UAT).

Subsequent Steps

The Release 1 software met the minimum compliance mandate within six and a half months. The team of roughly 80 people started requirements for release 2 during release 1 UAT. They shifted to a 3 month release schedule, delivered 2 additional releases in 2013, and by early in 2014 were working on release 4. At each release, the business prioritized and chose the scope most critical to them. Therefore, each release delivered the highest business value more rapidly than experience with traditional approaches.

Visibility and Adaptability

Stakeholder management became much easier because the team made changes early in a release as needed. Daily burn down charts showed the quantity of story points delivered. Stakeholders knew immediately when the burn down trended negatively. Similarly, the team was fully accountable to each other on a daily basis. They analyzed situations in real-time and corrected plans, by sizing stories differently or adding or changing resources. Business owners reported that they had never had so much visibility into one of their projects before.

By mid-year, the Agile-TDM approach had proven value – the enterprise application met the compliance mandate in record time. The team confidently requested additional funding for the next releases, which began to roll in every three months.

Despite around 14 years of Agile experiences, this article points out that most approaches are missing perhaps the most important ingredient in software, TDM. Integrating TDM with Agile significantly advances the pace and quality of software development. It does so by separating the business logic (formerly known as business rules) within the software. This enables the team to divide it into pieces, specify it, and validate it to find logic errors, and all of this prior to testing. From here, the team tests it to determine if it is the correct logic for the business – that it leads to appropriate conclusions. All of this happens smoothly within a rapid iterative development scenario. In fact, rapid iterative development of business logic delivers near perfect logic from a purely business perspective.

This article also identified 5 common criticisms or challenges of Agile, along with enhancements to mitigate them. Figure 4 summarizes the enhancements.

Figure 4: Three Techniques for TDM and Agile

Finally, a real-world case study proves that TDM and Agile are ideal together. Each one enhances the benefits of the other, with less risk, greater business value, and improves the business-IT partnership. In In fact, the Agile Approach in this article facilitates the successful adoption of an Agile approach at the enterprise level.

Author: Kedar Kulkarni (edited by Barbara von Halle)

Kedar Kulkarni is a Decision Architect at Sapiens and an Agile Trainer. Kedar leads projects using Sapiens DECISION Suite of products (The Decision Model- TDM is the cornerstone of the DECISION Product Suite) , with an integration with the Agile approach. He is a founding member of the IIBA Pune chapter in India and a member of the Harvard Business Review Advisory Council.

Kedar has, over the past 17 years, led and delivered multi-million dollar change management projects with in different industry domains such as Banking, Mortgage, Trading/Exchanges, Manufacturing, etc. across North America, Europe and Asia. He has taught at leading business school programs in India and also ran executive education and training programs for corporations in the subjects of Business Planning, Corporate Strategy, Business Analysis and Agile Methods. He can be reached at [email protected].

Barbara von Halle, subsequent to the acquisition of KPI by Sapiens, consults with Sapiens on The Decision Model and The Event Model. She is co-inventor of The Decision Model, The Event Model, and co-author of The Decision Model: A Business Logic Framework Linking Business and Technology published by Taylor and Francis LLC 2009. Her earlier book, Business Rules Applied (Wiley) was a Jolt Award finalist from Software Development Magazine.

She began her career in data management, consulting to major corporations on enterprise data management. As The fifth recipient of the Outstanding Individual Achievement Award from International DAMA, she was inducted into the Hall of Fame in 1995. Her first book, Handbook of Relational Database Design continues to be a standard reference in database design. She was the most popular columnist in the leading publication, Database Programming and Design magazine for over five years.

She holds a BA degree in Mathematics from Fordham University and an MS degree in Computer Science from Stevens Institute of Technology. She can be reached at [email protected] or [email protected]. For information on Sapiens’s DECISION (enterprise software for full life cycle decision modeling and deployment), see www.sapiensDECISION.com.

Reference:

1. The Decision Model: A Business Logic Framework Linking Business and Technology, von Halle & Goldberg, © 2009 Auerbach Publications/Taylor & Francis, LLC.

2. http://www.bankersonline.com/security/bsapenaltylist.html

3. For an example of “stub”, see http://www.modernanalyst.com/Resources/Articles/tabid/115/ID/3220/How-to-Introduce-The-Decision-Model-Using-a-Real-World-Compliance-Example.aspx

4. Also referred to as the Zachman Framework, considered the basis of all Enterprise Architecture, which is a Copyright work of John A. Zachman of Zachman International. A detailed explanation of the Framework may be found in The Decision Model: A Business Logic Framework Linking Business and Technology, Chapter 13. (von Halle and Goldberg, Auerbach NY 2009)

5. Because an Agile iteration delivers a piece of working software, other models and deliverables for remaining rows (i.e., Row 3-5) may be used by technical team members to correlate Row 2 models to executable code.

6. Silver Level 1 is a basic working set of shapes almost entirely familiar from traditional flowcharting. The Level 1 and Level 2 palettes correspond exactly with the Descriptive and Analytic subclasses of the final BPMN 2.0 specification. Level 1 covers the entire Descriptive subclass.

Silver, Bruce (2012-01-12). BPMN Method and Style, Second Edition, with BPMN Implementer's Guide (Kindle Locations 565-567). Cody-Cassidy Press. Kindle Edition.

7. www.irise.com