Presentation: TweetHigh Throughput, Horizontally Scalable Workflow

Do you need to react to millions of events in near realtime? Do you need to correlate events based on business keys such as User ID or Order ID?

Do you need to process these events with complex logic based on sequence counters, fork-join, mutual exclusion or timeouts?

Then you will already have tried existing workflow engines but found that they were unable to deliver the performance you require yet were still hard to use and bloated with features that you didn't need.

At Camunda, Workflow and Business Process Management are part of our DNA.

Companies around the globe run mission critical processes on our current open source platform. Now, we are rebuilding the workflow engine from the ground-up with high throughput and horizontal scalability as our key focus. Our design supports high levels of concurrency and will enable stateful event processing on a completely new scale.

All this is built on simple concepts and an easy to understand architecture. It separates event correlation, core workflow execution from business process management aspects such as human task management, auditing and reporting. It allows you to use as few or many of these components as are actually needed.

In this talk Daniel Meyer will present our progress so far. He will show how distributed event correlation can be used for localizing concurrency conflicts like event ordering, synchronization and mutual exclusion. He will also show how conflicts can then be resolved using state machines in a lock-free way.