Process modeling is a fundamental part of every business analyst’s work, used by business managers to understand their operational procedures and plan performance improvements. They are often also a communications tool between business and IT teams when discussing system changes and upgrades. However, the problem is that these models are almost always incomplete, imprecise or incorrect, as they only represent a human interpretation of what the process does, or what the related systems were originally developed to do. They do not show what the process actually is, or how it is deviating from best practices. Process mining, on the other hand, directly captures the actual process behavior based on the log files of systems that people use to perform work, turning tacit knowledge into explicit knowledge. This article explores the complementary nature of both process mining and process modeling as important tools and techniques for accelerating analytical efforts.
Analytical techniques in process discovery and improvement, including process mining and simulation, have been available for many years. In the past, however, they were used primarily by people with a technical analytical background. The current business environment and technological advances have pushed these analytical tools into the mainstream, where they are being recognized as essential for all levels of business analysts.
What’s driving the need for analytical techniques in process improvement
Although we tend to attribute all business and technology changes in these times to the pandemic, there are many complex factors involved in this move to analytical techniques for process improvement.
Work has become more automated. Many business processes are supported completely by computer platforms, although they can include multiple systems in a single process. Some are automated, and the non-automated steps are delivered to a worker using the technology platforms. This means that most of the activities in these processes are captured in some way by a system, creating a record of work passing between people and of tasks being performed at each step.
Work has become more changeable. The pandemic forced many businesses to shift their business models and rework their disrupted supply chains, but this was already occurring due to globalization and competition. New nimble competitors are creating business models that challenge incumbents with the ability to change their processes and supply chains frequently to adapt to changing conditions.
Work has become more distributed. Processes have changed drastically, supply chains have been interrupted, and many processes now include workers distributed across multiple locations. This is due in part to the pandemic forcing remote work and closing physical facilities, but also to globalization and the need for flexible supply chains to serve changing business models. A single process that used to occur within one office may now have participants across the country, or even around the globe, and managing these distributed processes requires additional coordination and oversight.
Work has become more complex. With greater automation, constantly changing processes and wider geographic distribution, an end-to-end core business process can be too complex and too volatile for any single person to have a detailed understanding of it. Static process documentation based on manual information gathering will not reflect the current state, or be useful for process improvement. Furthermore, manual analysis methods such as job shadowing and in-person interviews are hampered by a distributed workforce. Business analysts traditionally use these techniques to discover and document processes, but with workers in multiple geographic locations, exacerbated by pandemic restrictions on personal interaction, this is often not possible.
In short, today’s business processes can’t be properly analyzed and improved using only traditional manual methods: semi-automated analytical methods including process mining and simulation must be combined with manual methods to get the full picture.
Differences of “as -is” and “what-if” process modeling techniques
Process modeling is a fundamental part of every business analyst’s work, used by business managers to understand their operational procedure and plan performance improvements. They are often also a communications tool between business and IT teams when discussing system changes and upgrades. However, the problem is that these models are almost always incomplete, imprecise, or incorrect, as they only represent a human interpretation of what the process does, or what the related systems were originally developed to do. They do not show what the process actually is, or how it is deviating from best practices.
Process mining, on the other hand, directly captures the actual process behavior based on the log files of systems that people use to perform work, turning tacit knowledge into explicit knowledge. This can be used to validate – or even generate –process models, identify conformance issues that increase business risk, and model performance optimization. Process modeling and process mining are complementary, not competitive, techniques that help create the important “as is” and “what-if” process modeling and simulation techniques.
Let’s explore a typical scenario. The key focus for many business analysts is to create business documentation using process modeling tools and text-based description, rather than performing in-depth analysis of business processes. Documenting current processes and procedures in this way has value, but it’s not really analysis, and it is unlikely to discover opportunities for radical improvement.
Business analysts gather information through observation: job-shadowing to document task procedures and the human-computer interactions, interviewing teams about operational processes, or viewing operational metrics. Because this is based on only the worker use cases plus subjective interpretations of the overall process, the result is an incomplete – or even incorrect – view of what actually happens, and little insight into how this links to overall business performance.
To complement their manual documentation and analysis tools, business analysts can use process mining for creating an “as-is” model of processes that are supported and automated by enterprise systems, since the activities of those processes are captured in the log files of the systems involved. The advantage of using process mining over manual process modeling is that it’s data-driven: the processes discovered are the actual operational processes, including automated activities and human-computer interactions, not an interpretation of those processes. This includes statistical information on the frequency and duration of each activity and pathway in the process, allowing for much more detailed analysis based on the data.
But process mining can go far beyond just creating a model of the current-state process. Because it’s based on the actual record of many individual transactions and may combine data from multiple systems, it allows an analyst to explore the model in ways that aren’t possible with static process models. Process mining tools show all the different ways that the process actually executed, and help identify the reasons for the variations from the expected process. Data-driven analysis techniques including conformance checking and root cause analysis are then used to find the problems in the processes, and what is causing them. This locates problems such as places in the process where compliance controls were bypassed, or where the same transaction unnecessarily repeats a loop between the same group of workers several times before completing.
Process mining tools can also be used to explore “what if” scenarios for predicting what would occur if the process changed in some way:
- Where would bottlenecks form if our throughput volume was drastically increased?
- What resources would sit idle if it was drastically decreased?
- If two parts of the process ran in parallel, how much would that decrease the cycle time?
- If a human task was automated using robotic process automation, how much time and money would that save?
Since data about time spent by workers in the process is captured in the model, this can translate directly to calculating trade-offs in costs and cycle time, and figuring out the best way to allocate resources within processes.
Past generations of process mining tools were used primarily by technical analysts with a deep understanding of the underlying algorithms. This created a barrier for business-oriented analysts who have a deep understanding of the business, but less formal mathematical or technical background. The current tools, however, make the capabilities much more accessible and usable by a broader audience: users still require analytical skills and process knowledge to guide the analysis and contextualize the results, but don’t need to be a data scientist to reap the benefits. Process mining is no longer a tool for specialists, but a self-service capability for any business analyst.
The ideal scenario for a business analyst, then, is to combine their knowledge of business and their manual documentation skills for task procedures with the powerful analytical capabilities of process mining and simulation. This allows them to create a complete and accurate picture of the current state – including all of the variations and potential compliance risks – and detect where changes to the process can have the most beneficial business impact. Having an understanding of the manual procedures for individual workers against the background of the overall process flow also allows them to suggest additional areas for automation of routine activities.
Traceability and the benefit to Business Architects
Although process mining is a powerful tool for business analysts as described above, data-driven analytical tools have direct benefits to business architects.
Business architects are concerned with the traceability of business objectives: being able to identify top-level business goals, map those to departmental or business capability goals, then directly to metrics at the operational process level. Once executives agree on corporate goals, that traceability ensures that processes are designed to work towards those goals, and the front-line workers are being rewarded for performing tasks that contribute to the goals. Without that, those goals will never be achieved, since no one will be doing the actual work to make them happen.
Traceability is also important from the bottom up: the tasks that the workers are performing, and the detailed steps of the operational processes need to map upwards towards corporate goals. If that traceability isn’t there, then there are likely operational processes and worker tasks that are completely unnecessary, hindering productivity.
Process Mining for Operational Management
The analysis provided by process mining can have far-reaching impacts for operational managers since it provides total transparency into how their processes actually work. With broad knowledge of their own business area, operational managers can interpret the analysis for the purposes of continuous improvement beyond just process improvement:
- Conformance checking, which compares what’s actually happening in a process against what is expected, can be used to identify training opportunities for certain roles or workers.
- Cost/resource trade-offs can be used to make decisions about outsourcing.
- Expected bottlenecks under increased load can be used to make decisions about staff level increases during seasonal peaks.
- Identifying sources and cost of waste can be used to improve quality management and reduce defects.
- Process transparency and conformance checking is also useful for supporting compliance audits. Auditors can look at the frequency of different types of compliance violations, and drill into specific transactions to understand the root causes.
Summary
With today’s highly complex and volatile processes, organizations need much greater analytical strength to understand how their business actually works, and how to improve it. These processes are too complex to analyze using manual methods alone, and business analysts need to use data-driven analytical methods such as process mining in combination with manual methods for the best results.
Having data-driven analysis as part of business process analysis also allows business architects to determine goal traceability, and provides managers with a rich set of information for making operational decisions.
Author: Sandy Kemsley
Sandy Kemsley is a technical strategy consultant specializing in digital process automation. With many years of hands-on systems implementation experience, she works with large organizations on their digital transformation strategy and implementation projects. She writes a popular process management blog at column2.com, is a featured conference and webinar speaker on process automation and digital transformation and is a contributing author to books on social BPM and adaptive case management. Sandy is recipient of the 2019 Workflow Management Coalition’s award for Outstanding Business Transformation Consultant.