The Flow Director Architecture - easy to use and it scales!

07 Jan 2020 | Andreas Müller

If you see Flow Director the first time, either by using it or by watching some of our clips, you don't see where the real power comes from - it is the 1000 HP engine behind.

Based on SwiftMQ

SwiftMQ is the platform where Flow Director runs the flows. It is very important to understand that we do not have our own, self-developed runtime but use a proven, mature and high performance open source messaging system.

A SwiftMQ broker is called a router because it routes messages from producers to consumers. A single router can handle a lot of load but you can connect routers as a network called Federated Router Network to scale or to build dedicated topologies like this:

Router Network

Each router has connectors for JMS, AMQP and MQTT so it supports these connections natively.

SwiftMQ Streams

SwiftMQ is an outstanding messaging system but what makes it really unique is the integrated micro service platform SwiftMQ Streams which allows to intercept any internal message flow of a router and process it. These Streams can be created with any scripting language like JavaScript, Groovy, Scala. They work against the Stream Interface which provides all necessary components to intercept, analyze and process the message data. Streams are deployed and run inside a SwiftMQ router, are compiled to byte code and therefore high performance. A single router can run hundreds of them.

Flows

Flow Director works with flows which are graphically connected components. These flows are stored at the Flow Director server in JSON files.

A flow has the following lifecycle:

  • deploy
  • activate
  • deactivate
  • undeploy

Deploy

During deploy, Flow Director takes the JSON that contains the flow declaration and on the fly generates JavaScript code that forms a SwiftMQ Stream. It then sends it to the router node which stores it in a local persistent queue-based repository. Flow Director also creates the Stream configuration that is part of the router configuration.

The generated code actually creates 2 Streams. One is the flow itself, embedded in a small runtime that allows runtime changes like enable debugging or throughput counts and the other is a Stream that implements a shell which is used for the communication with Flow Director. So a flow can be actively managed by Flow Director.

Activate

During activation, the Streams containing the flow and the flow's shell are enabled and thus started. They are connected to the App environment (i.e. the App's log) and can be managed through the flow viewer.

Deactivate

The Stream is disabled.

Undeploy

The configuration and the repository entry is removed from the router node.

Flow Orchestration

Per default Flow Director comes with a single SwiftMQ router installed. However, you can deploy as many routers as you like and build any topology. You can run Flow Director / SwiftMQ fully separate but can also connect Flow Director to an existing router network without any configuration changes. Flow Director does everything automatically by deploying Streams that build the Flow Director and App runtime in the background. It detects when new routers join the network and starts an automatic provisioning process.

Once Flow Director is connected to a SwiftMQ router network, all nodes of that network are at your disposition like a grid. You can deploy flows on any node and they can communicate with each other through queues and topics. This is all done from a single Flow Orchestrator panel:

Flow Orchestrator

Summary

Flow Director is based on SwiftMQ and the SwiftMQ Streams engine. SwiftMQ can build any router network topology which is fully managed by Flow Director and at your disposition like a grid. Flows are generated to JavaScript code that is deployed on one or multiple SwiftMQ routers. The code is compiled to byte code and runs within the JVM of the router. A single SwiftMQ router can run hundreds of flows while a network of routers has no real limits.

Photo of Andreas Müller

Andreas Müller, CEO & CTO

Andreas is a well-known messaging expert, creator of SwiftMQ and in his side job CEO of IIT Software GmbH. He leads the overall development of Flow Director and has an eye on any piece to fit into the whole picture.

Data Privacy

Copyright ©2020 Edge Broker GmbH. All rights reserved.

Some icons by Icons8.