Cribl

Source

Kafka

Event streaming platform for processing, storing, and reprocessing streamed data

Apache Kafka is an open-source, distributed event streaming platform widely used for high-performance data pipelines, streaming analytics, metrics collection and monitoring, log aggregation, data integration, and mission-critical applications. As a durable message broker, Kafka enables applications to process, persist, and reprocess streamed data.

How to get data flowing

This is a built-in integration between Cribl Stream and Kafka.

Kafka as Source and Stream as a destination

  • Create a Kafka topic, write to it, and export to LogStream via Kafka Connect.

  • Configure Stream to read data from Kafka via Sources > Kafka.

  • Specify the Kafka brokers, topics, and optional settings (Confluent Schema Registry, TLS certificate, and SASL authentication parameters).

  • Stream will start fetching data as KDS streams become available.

Kafka as Destination and LogStream as a source

  • Configure LogStream to send data to Kafka via Destinations > Kafka.

  • Specify the Kafka brokers and topic to write to, along with other settings (record data format, compression and backpressure behavior, and optional Confluent Schema Registry, TLS certificate, and SASL authentication parameters).

  • LogStream will start sending data as it becomes available.

View Documentation

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on-Sandboxes for those interested in how companies globally leverage our products for their data challenges.