Apache Kafka

Integration with Ease.

Get Flow Director

Apache Kafka Integration

Integration with Kafka: Page View History Creation in Flow Director

Explore how Flow Director seamlessly processes page view messages from Kafka and generates historical data for visualization.

Integration Details:

  • Page View Handling:
    • Receive page view messages from a Kafka topic.
    • Establish a history of these page views for chart representation on Flow Director dashboards.
  • Kafka Consumer Operation:
    • Utilize a Kafka consumer that employs a synchronous API, requiring polling.
    • Within the polling interval, the Kafka consumer retrieves all messages received.
  • Flow Configuration:
    • The flow starts with a timer triggering a Kafka consumer on the PageView topic to obtain messages.
  • Message Processing:
    • Each message is initially in JSON byte format.
    • Two processing steps are performed: converting bytes to text and extracting JSON fields.
  • History Creation:
    • Processed results are directed to a history component.
    • Histories are automatically generated at minute, hour, day, month, and year intervals.
  • Chart Visualization:
    • These histories can be seamlessly linked to group charts in Flow Director.
    • This enables real-time visualization of page views over time.

Flow Director effortlessly integrates Kafka consumer and producer elements in various configurations. Once integrated, further connections to other messaging systems are facilitated.

Experience efficient and dynamic page view history creation and visualization using Flow Director's capabilities.

Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. It can be easily integrated into Flow Director and with any other messaging system.

Share this page