Execution Pointers vs Dependency Trees

Peter Harrison - Aug 1 '23 - - Dev Community

Who is this article for?

In this article I describe limitations of popular Business Process Management solutions, specifically around the difficulties of changing process definitions on the fly and the ability to modify context data and replay task execution without rewinding execution pointers. I describe how Gravity Business Automation uses Dependency Trees and Events to address these limitations.

BPMN and Execution Pointers

There are a raft of business process automation engines which allow you to wire together actions and decision points to design a business process. These include JBPM, Activiti, Bonita and numerous others. What they all have in common is a process description language called BPMN 2.0.

Provisoning Process

Provisioning Process Definition

In Business Process Model and Notation (BPMN 2.0) the process definition is described as linked tasks, decisions, splits and joins. A task is some action to take based on the data within a process context. Every process instance has its own context which contains the data for that instance.

The process engine follows a process definitions much like a computer program, following the flow from task to task. To track where it is there is an execution pointer. The execution pointer is key in tracking where the process has reached. Splits and joins allow for there to be parallel execution pointers, but for these to function explicit splits and joins need to be specified in the process definition.

I've had some experience with this style of tool, and as a result came to know some limitations.

The first is that if you have a long running process, such as provisioning a new customer which requires services to be started and hardware sent, you might have several hundred process instances in flight at the same time. If you need to modify the process design you cannot easily transition the existing in flight instances into the new process definition.

In fact what we ended up doing was allowing in flight instances to run through the old definition, and have new instances follow the new definition. Having two active definitions however was confusing and created its own issues.

The second issue was with exceptions. You can introduce explicit exception handling within a process definition, but they the primary process becomes concealed by a thicket of exception handling contingencies. Even then common issues could not be easily handled.

For example, imagine you collected the wrong email address from the customer, and the process has sent several emails to the wrong email address, all the while following the process definition and performing other tasks. How can you correct the email address and have the automation resend the emails without repeating all the other intermediate steps?

The inability to modify context data and replay explicit tasks without resetting a execution pointer and causing unwanted tasks to repeat caused process instances to fall off the rails. We had to go to a manual process for those that fail.

This resulted in terrible success rates as any minor deviation from the ideal process would tend to break in ways that made it impossible to continue a process instance.

Dependency Tree

Dependency Trees and Event Driven Orchestration

The Gravity Workflow Automation does not use BPMN process definitions. Instead it uses a dependency tree of tasks and an event driven approach.

Dependency Tree

Dependency Graph for Provisioning Process

In a dependency tree each task has trigger criteria that include the successful execution of other business rules or criteria involving the data itself. By creating dependencies between tasks we create a similar network to a BPMN diagram, only you don't need explicit connections or splits and joins to support parallel processes. Because all rules are evaluated at the same time, multiple rules can be executed in parallel.

You can modify a business rule at any time and it will take effect right away. You could for example introduce a new rule which sends a email to the customer, and on the next event the rule would trigger and send emails to the customer regardless of what stage the process is at. It is possible to constrain activation of course to ensure it is only sent at a particular stage if required.

This means process changes can take effect right away consistently across all the process instances, rather than only taking effect for new process instances.

It also means that you can modify data within a process instance and the event execution, and then rerun particular tasks of your choice. If necessary you can correct data and rerun automated tasks. Because there is no process pointer you don't end up rerunning tasks that are already complete.

Because all the business rules are evaluated on each event there is no need for an execution pointer. Everything runs in parallel by default.

Data Update to Task Execution

The core mechanism for automation in Gravity Business Workflow Automation is a cycle. Incoming data insertions or modifications result in an event being raised. This is added to a queue.

Data Update to Action Pipeline

Data Update to Action Execution Cycle

The events are read from the queue. Each event has a related data source, which in turn has related business rules. If the business rules trigger criteria is met there a sequence of actions is executed. Results from each action is supplied to the next in the chain. Each action may result in further modifications to the related data, which in turn begins the cycle again.

In a Gravity cluster each node consumes events, and so the execution of business rules are spread across the cluster automatically. Just because an API is received and data is updated from one node does not mean the processing is performed by that node.

Gravity was designed to be scalable, in that it is possible to scale up the cluster nodes to handle load.

Human Interaction

Traditional BPMN includes explicit human tasks which treat staff as components within a process definition. This means they can only contribute data at specific narrow opportunities. This works well for computer programs, but it is it undermines the flexibility of humans.

In Gravity we usually allow staff to modify data at any time. It is possible to restrict access if necessary. Each change in the data can cause additional tasks to run. Staff can also revert tasks, which means when the events are run the tasks will be rerun. This gives staff the ability to fix issues and get automation back on track when things go wrong.

Development of Gravity was a consequence of addressing the issues in common between all the execution pointer based automation systems.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .