This is an article about decision logic and the ways in which it can be analysed and harnessed in the requirements gathering, design and testing phases of implementation projects. There are two areas of special focus here:
- The Decision Model (TDM), which is a normalized approach to business logic definition which I have been using in implementation projects for the past years, and
- SAP, since I am an SAP delivery architect working in financial services industries and in particular the SAP implementation methodology ASAP for Financial Services Industries (FSI).
However, the approaches, techniques and experiences shared here are applicable in a wide range of contexts, and you don’t have to know about TDM, SAP or financial services to find the experiences here relevant to your situation. What you do need is curiosity about better ways of defining and testing business rules, and a willingness and opportunity to apply lessons learned to your own advantage and in your own specific work situation as an analyst, whatever that may entail.
This article looks at practical experiences of implementing business rules using TDM and SAP from several angles, while also raising some of the questions which I find asked most frequently and insistently in my work, such as:
- Why do I need anything other than an existing rules engine to define and manage business rules?
- Why would I want specialized tooling for business rules when I already have tooling for requirements gathering?
- How does improved business rules definition help me with testing?
However, before getting into questions such as these, I would like to start at the beginning by explaining how business rules became a problem during my work as an SAP architect, how my colleagues and I addressed it and how this led to the adoption by SAP Financial Services Industries of The Decision Model as their official standard for dealing with definition and testing of business rules.
Industrializing Delivery in SAP Financial Services Industries
First, here is some background on SAP. SAP is a global leader in providing and implementing solutions for banking, insurance and other financial services industries. This is confirmed by analysts such as Gartner and Forrester, who constantly publish assessments both of SAP product capabilities and of the extent of SAP’s market penetration compared with other vendors.
To achieve and sustain this leadership as a relatively new entrant to the FSI space over the past fifteen years, it has been vital for SAP to demonstrate not only vision in terms of product, but also ease of implementation in live projects. Analyst reports confirm the progress made by SAP in this area, for instance in the Gartner magic quadrant assessments for core banking, which showed SAP shift from leadership in completeness of vision in 2008 to leadership in ability to execute by 2012.
One of the keys to achieving this recognition has been to industrialize delivery by means of a standardized delivery methodology, known as AcceleratedSAP 8.0 for Financial Services Industries – or ASAP for FSI for short. Key takeaways about ASAP for FSI are that it:
- Takes several orthogonal views as starting points – business process view, business capability view, systems integration view – and brings them together to provide a single process / service model and a single set of functional design deliverables.
- Provides standard deliverables and templates with business content, covering requirements gathering, design, build and test.
- Is reinforced by a worldwide training program for SAP and partner consultants which has ensured hundreds of staff trained in a single approach.
- Is supported by standardized tooling with business content (templates etc.) using SAP PowerDesigner.
- Is subject to ongoing innovation e.g. to provide agile variants, variations for downstream analytic systems as well as operative (e.g. core banking) systems.
So far so good. But where do business logic and business rules fit into the picture? This is the real focus here. So, let me explain ….
The Problem with Business Rules
When the SAP team was developing the ASAP for FSI method, we went to great pains on our pilot implementation projects to perfect the deliverables used, develop the templates needed, provide examples for other projects and for training, and prove the approach in real live situations. By the time we had finished, we had a series of deliverables and approaches which covered the different domains needed:
- High level and detailed application architecture and integration design
- Data models (UML)
- Process models at both pure business and systems levels (BPMN, UML)
- Functional requirements definitions
- Functional design definitions
- SOA-compliant service model definitions for all integration patterns, fully semantically harmonized
- Testing deliverables
But – and it was a big but – there was one area where our efforts were not, in all honesty, really successful. This was the area of business rules. Experienced readers will likely concur that the area of business rules has historically been somewhat unsuccessful. Here are the kinds of problems which we encountered and which most readers have probably also encountered:
- Free-text business rules took months to capture initially, resulting in delays to requirements gathering.
- Traceability of business rules to process models and functional design documents was laborious to implement.
- Fixing incomplete and incorrect business rules led to heavy rework cycles, since updates to the business rules definitions had to be followed by updates to the process and functional design documents on which they depended.
- Testing of free-text rules involved manual replication of the business rules in test scripts and test cases.
These problems were not trivial. They slowed our progress and were rightly unpopular, regardless of the fact that the approach had been agreed to by all parties and was not specifically a problem for, or created by, SAP.
For the SAP team, this led to an intense search for a better way of doing things. We discussed with the client, with the system integrators, amongst ourselves; we proposed improvements; and we read – books, articles, blogs, whatever came to hand. One beneficial side-effect of this was that we came to a clear view of what was needed in an approach to business rules – of “what good would look like”. This is summarized in Figure 1.
Figure 1: A Better Solution
“If this seems like an impossible wish list, we have been here before.” (Barb von Halle)
As you can see from Figure 1, the core of the problem was the lack of structure in the free-text business rules we were using. Business rules were not structured in a way that made them easy to grasp top-down, using a hierarchy or taxonomy. And their usability in detail was severely constrained by the lack of atomic structure – free-text definitions can be defined precisely to an extent, as in a legal contract, but there is always undesirable room for interpretation. What we needed was an approach which provided both top-down readability and atomic precision.
The Eureka Moment
After months of searching for a better answer to the business rules question, one of the SAP team bought a book by Barb von Halle and Larry Goldberg entitled The Decision Model [1]. I borrowed the book and read it with detached curiosity until I came to the diagram in Figure 2.
Figure 2: Eureka Moment
On seeing the diagram on the left, I immediately recognized the exact problem with which we had been grappling – or at any rate, one essential aspect of it: the complexity and poor maintainability of process models which attempt to encapsulate both procedure and business logic in a single model. The solution proposed was breathtaking in its power and simplicity: take out the business logic, leaving the process model as pure procedure, and put the business logic into a portable, top-down readable model of its own – a classic separation of concerns, which SAP had applied so effectively in other areas such as service-oriented integration. I had no sooner seen the solution than I was sold on the concept.
Understanding the detailed workings of The Decision Model took slightly longer of course. How it works in essence is this. TDM takes the underlying “if … then …” statements (atomic logic), and puts them into a “Rule Family” table as:
- condition (e.g. “if policy pricing is not within bounds …”) and
- conclusion (“… then policy renewal method is manual renewal process”):
See Figure 3.
Figure 3: Detail of Example, Decomposed
TDM then resolves the familiar programmers’ problem of not mixing “ands” and “ors” by arranging implicit “ands” across multiple columns, while “ors” are arranged into separate rows as shown in Figure 4.
Figure 4: Solution to the Problem of "and" and "or" in The Decision Model.
Finally, TDM arranges the Rule Families into a hierarchy, and makes the hierarchy as a whole the overall Decision Model – portable, transparent and logical as required. I will show an example of the hierarchical organization in the section following.
First Steps with The Decision Model
Having identified The Decision Model as the prospective solution to our business rules problems, the next step was to test the theory on some real-life examples. The first test case was a decision around core banking account closure. We chose this because it is complex enough to test the value of TDM without being too large a problem to be manageable. The input data was a set of 82 free-text rules. Figure 5 shows the kind of structure and organization involved, albeit through a generic example showing 10 lines only.
Figure 5: Free-text Rules Defined for Account Closure Decision (generic example)
As you can see from the callouts, the rules as defined in free-text format suffer from a lack of structure. Because of the absence of a top-down taxonomy, decision and procedural rules are mixed together, and rules for different purposes (locks, hub/spoke products, etc.) are not grouped together. The atomic logic is also not clear - “ands” and “ors” are not separated for instance. Contrast this with the view in Figure 6 of the overall decision model which emerged after analysis (generic representation, not the original customer example)
Figure 6: Decision Model for Account Closure Decision Abstracted from the Existing Free-Text Rules (generic example)
The beauty of The Decision Model solution, to my mind, is its clarity. Just looking at the model itself, you can immediately see the inherent structure and content of the decision – in this case, that
- the account must not be locked;
- there must be no pending items (the balance on the account must be zero);
- there must be no dependent, linked products to which access would be lost if the hub account (in the hub and spoke model used here) were closed.
However, there are a number of other significant advantages which this overview highlights:
- Transparency of structure – the top-down readability just referred to
- Atomicity of logic – you can see all the variables involved, and drilling down to the underlying rule family tables gives you the complete picture
- Maintainability – because the logic is defined atomically and because the hierarchical structure is easy to alter if necessary
- Portability & reusability – you should only define the data variables once and they are then reusable elsewhere
- Radically simplified cross-references – just consider the simplification involved in being able to remove numerous separate rule references and replace them with a single decision reference
- Reduction in paperwork overheads - because the models and cross-references are so significantly simplified
- Business ownership – experience shows that the business find the TDM presentation easy to understand and own
- Simplification of testing – this is a whole topic on its own.
The benefits which flow from these advantages are equally striking:
- Reduced delivery cost / timescales – simplification of models simplifies the overall implementation effort
- Greater certainty of delivery – because it is finally clear what the rules really are and what they mean
- Empowerment of business relative to IT – because business can easily take ownership of the rules based on a clear logic model
- Potential automation and test generation for further benefits – a point for later.
From Pilot to Production
The pilot which I have just described showed the way forward, but it came too late to affect the immediate implementation to which it related. So it was not until the next core banking implementation project that the opportunity presented itself to use the TDM approach from the start, from requirements gathering onwards. Here, we agreed from the start that the TDM approach would be adopted. The draft method dependency diagram included TDM, as shown in Figure 7.
Figure 7: Draft Method Dependency Diagram where TDM Business Rules are Formally Recognized
From here on in, the idea started to take off, both within SAP and outside it. Within SAP, it was agreed, based on the successful transition to production of TDM on a live project that it should be incorporated into the existing ASAP for FSI method; so we moved on to training and disseminating the approach more widely within SAP for FSI.
Outside SAP, meanwhile, the idea seemed to take on a life of its own. For instance, the initial customer where TDM was adopted on the SAP project soon decided to apply it more widely, including on projects where non-SAP core banking packages were being implemented. It just goes to show that there really is no copyright on good ideas!
Objection!
When introducing The Decision Model concept and approach to a new audience, there are many questions which get raised. Usually, people quickly get the point that TDM offers significant benefits compared with more traditional approaches. However, they usually move on quickly to ask searching questions about how TDM fits in with existing tooling and applications handling business rules, and here a different angle on the topic is needed. The two questions which crop up most frequently are these:
- I already have a rules engine (Business Rules Management System or BRMS), so why do I need TDM in addition?
- I already have tooling for requirements gathering, so where does TDM fit in the tooling picture?
I don’t propose to cover these topics here in the depth. However, it would be advisable to provide at least short answers to these questions right now.
Objection #1: I already have a rules engine
Business rules engines (BREs) or Business Rules Management Systems (BRMSs) are a well-established part of the systems landscape in financial services and other industries, and the benefits which these provide are widely acknowledged and well documented.
SAP has its own BRMSs – SAP Business Rules Framework Plus (SAP BRFplus), SAP Decision Management System (DMS) and SAP Business Rules Management (SAP BRM) are the best known. But there are numerous non-SAP products in this space as well, including FICO Blaze Advisor, RedHat / JBoss Drools, TI~BCO ActiveDecisionsMatrix and IBM Operational Decision Management (previously ILOG) to name some of the leaders.
Most if not all of these BRMSs work on principles which have some points of comparison with TDM. All those that I have analysed make some use of decision tables, somewhat similarly to the rules families of TDM, and some of them also provide templates which are intended for use to communicate with business users, which is one of the greatest strengths of TDM itself. So, what is the distinction here, and is there a co-existence model which could be proposed?
The first comment to be made here is that the overall purpose and focus of BRMSs is different from that of TDM. The primary intention behind a BRMS is to execute rules operationally. This is an IT focus, not a business one. For this reason, the screens which are used to maintain BRMSs are not built with communication to a business audience in mind. This is an observation, a fact if you like, certainly not a criticism. Even where templates for communication with a business audience are provided (as with TIBCO ActiveDecisionsMatrix and JBoss Drools, for instance), these are a spinoff from the central systems purpose, rather than being the main focus of the application.
But you can go further. The fact that templates for communication with a business audience (e.g. using decision tables) are available to accompany a particular BRMS, does not give those templates the same level of usability in a business context as TDM itself. On the contrary, there are two essential features of TDM which set it apart completely from the BRMS templates:
- Normalisation of atomic logic definitions – the handling of the AND/OR problem, the completeness of definitions in TDM. TDM has a scientific basis which is lacking elsewhere.
- The top-down structure, which provides readability, portability, maintainability etc. If you look for top-down definition in the BRMS structures, you will find it – but not in a format which is clear, transparent and communicable to a business audience.
In a single sentence, TDM has a transparency and completeness which is ideal for business audiences, whereas BRMSs have an ability to execute business rules in a systems context, and the overlap between the two is only limited. It is for this reason that the tooling associated with TDM, Sapiens DECISION, has been used with several of the market leading BRMSs to feed rules which have been defined in TDM into a BRMS for execution.
Objection #2: I already have a tooling for business rules
This leads neatly to the next point, concerning the relationship of TDM to existing tooling for requirements gathering and business rules definition.
In situations where an enterprise has already bought licenses for requirements gathering, and this includes support for business rules definition, there will be financial and other pressure to use the tooling already in place to record and govern business rules. There are two overall rules of thumb here:
- It is certainly possible to store business rules in industry standard tools (CA ARIS is a common example) and use version control and other facilities to maintain TDM business rules in a controlled environment.
- To get the full benefit from use of TDM, however, you need to use specialist tooling, capable of supporting the central glossary needed to promote reuse of TDM decision models and rule families, to check the completeness of TDM rules definitions, to generate a complete set of test cases based on TDM rules and so on. The only specialist tool which currently does this is Sapiens DECISION, though there is no reason in principle why other such tools could not be developed in future.
So, it is all a question of what you wish to achieve by using tooling. Just as one size does not fit all in core banking applications (different functionality is needed for loans as against deposit accounts or cash management arrangements), one size doesn’t fit all in requirement definition applications either.
TDM, BRMSs and Decision Modeling Notation (DMN)
There is one further point to make, as a footnote to both objections covered above. The normalized principles and structure of TDM, which differentiate it from BRMSs and their business communications templates, and which drive the need for specialized tooling, inspired the Object Management Group to establish a new standard, namely Decision Modeling Notation or DMN (see http://www.omg.org/spec/DMN/1.0/Beta1/). TDM is the first DMN-compliant model for business rules, but there will doubtless be others in future [2]. The differentiation between TDM and BRMSs would apply to any DMN-compliant business rules notation. As and when the BRMS vendors adopt DMN into their approaches and products, the gap which exists today between TDM and the BRMSs can be expected to narrow or even disappear. But that is still in the future at the time of writing.
The Onward Journey
In April 2014, SAP made an SAP FSI Method Announcement, to the effect that The Decision Model is being incorporated officially into the ASAP 8.0 for FSI method, which is an add-on to the overall ASAP 8.0 method used across all industries in SAP.
The journey to this point is summarized in Figure 8.
Figure 8: Journey to TDM Adoption in SAP Financial Services Industries
The Decision Model at SAP has many possible futures. Some areas which could be considered and investigated further are:
- Test automation
- Data quality management
- Standard logic libraries and
- Conversion of regulatory unstructured text to executable code.
Finally, a personal view on what the future holds. TDM is not a minor improvement in business rule management. It is a disruptive approach introducing full business logic modeling for the business.
“A similar disruptive solution from the past is the Relational Model. It changed forever the way we think, manage, and leverage data for business success. But it was not an overnight success. It had critics and stirred resistance. Nevertheless, its eventual adoption by software vendors and practitioners contributed to the advent of the Information Age and related advances in business maturity.” (Barb von Halle)
If history is a predictor, TDM is a powerful, yet simple, idea whose acceptance, cannot be controlled. Therefore, in organizations where you plant the TDM seed, someone will get it and run with it.
Authors: Barbara von Halle & Nick Illingworth
Barbara von Halle, subsequent to the acquisition of KPI by Sapiens, consults with Sapiens on The Decision Model and The Event Model. She is co-inventor of The Decision Model, The Event Model, and co-author of The Decision Model: A Business Logic Framework Linking Business and Technology published by Taylor and Francis LLC 2009. Her earlier book, Business Rules Applied (Wiley) was a Jolt Award finalist from Software Development Magazine.
She began her career in data management, consulting to major corporations on enterprise data management. As The fifth recipient of the Outstanding Individual Achievement Award from International DAMA, she was inducted into the Hall of Fame in 1995. Her first book, Handbook of Relational Database Design continues to be a standard reference in database design. She was the most popular columnist in the leading publication, Database Programming and Design magazine for over five years.
She holds a BA degree in Mathematics from Fordham University and an MS degree in Computer Science from Stevens Institute of Technology. She can be reached at [email protected] or [email protected]. For information on Sapiens’s DECISION (enterprise software for full life cycle decision modeling and deployment), see www.sapiensDECISION.com.
Nick Illingworth is a leading IT banking architect working for the services arm of SAP, the leading global provider of business software. A member of the EMEA management team for SAP FSI services, he is a co-author of the standard implementation methodology for SAP FSI (‘ASAP for FSI’), and was instrumental in introducing The Decision Model to SAP, pioneering its use and driving its recent adoption as a global standard for SAP in Financial Services. In a 25-year career working for Deutsche Bank, Barclays Group and IBM as well as SAP, he has been lead architect on projects in retail, corporate and central banking, and has particular focuses in the domains of core banking transformation and banking supervision. He is a frequent speaker on webinars and at conferences on topics around SAP, banking and business rules. Nick can be reached at [email protected]
References/Footnotes:
- The Decision Model: A Business Logic Framework Linking Business and Technology, von Halle & Goldberg, © 2009 Auerbach Publications/Taylor & Francis, LLC.
http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Dstripbooks&field-keywords=The+Decision+Model
- DMN tools are appearing in the marketplace.