Are you managing a Cribl environment? We love that for you; you’re at the forefront of complex data orchestration. As the steward of this dynamic data ecosystem, you have to manage and optimize the flow of information from diverse sources. As data volumes grow, the struggle gets even more real.

No worries, though. You’ve got Cribl Stream.

Monitoring Stream is critical. When you do it effectively, you can address issues, enhance operational efficiency, and maintain peak performance in your data strategy.

Elasticsearch is one way to do that, with its capabilities for quick searches, ability to handle large data volumes and AI-driven insights.

Whether you’re already an Elastic shop, are interested in custom dashboard functionalities, or want to explore some cool features, Cribl offers the flexibility to direct Cribl Metrics/Logs to your chosen destination.

Let’s get into it.

How?

• Send metrics to Time Series Data Streams (TSDS) for 70% less storage. Crank that up with the Downsample ILM action for even more disk space savings! 🚀✨
• Logs are saved in Data Streams.
• Link Cribl Internal Metrics and Logs to Elasticsearch with QuickConnect.
• We made dashboards and alerts for you—just import them into Kibana.

Getting Started

See how quickly Elasticsearch can help troubleshoot your Cribl setups.

This short demo only uses Cribl Stream, but full instructions are over on GitHub, covering:

• Elasticsearch preparation with Index Templates
• Cribl Pipeline for mapping conflicts and ECS support
• Populating leader logs for the included Dashboard

Cribl Configuration

1. Go to Stream > Manage > (Select a Worker Group) > Data > Destinations and enter elasticsearch

• Go to Elasticsearch and then Add Destination
• Set Output ID to cribl_elasticsearch
• Set Index or Data Stream* to `metrics-cribl-internal`
• Fill in the appropriate Bulk API URL and Authentication for your Elasticsearch cluster:

• Save and go to Data > Sources and enter cribl

• Go to Cribl Internal and then CriblLogs
• Go to Connected Destinations to enable QuickConnect with your Elasticsearch destination and the passthru pipeline:

• Go to Pre-Processing and set an __index field to logs-cribl-internal. (Optional: set a custom value in the custom_id field)

• Save and select CriblMetrics
• Go to Connected Destinations to enable QuickConnect with your Elasticsearch destination and the passthru pipeline. (Optional: add the cribl_metrics_rollup pipeline for reduced storage by aggregated metrics)

• Go to Pre-Processing and set an __index field to metrics-cribl-internal. (Optional: set a custom value with custom_id here too)

• Save and then commit and deploy the changes

Kibana Configuration

Dashboards, as well as Rules, can be imported with the Kibana UI:

• Download this file (link).
• Sign in to your Kibana space.
• Go to Management > Stack Management:

• Go to Saved Objects:

• Select Import:

• Click Import under the Select a file to import section and select the downloaded file:

• Click Import again:

You should now have imported all available Dashboards and Rules! If you’re running into issues here, note that the Kibana setting xpack.encryptedSavedObjects.encryptionKey may need to be configured.

The Demo

From Management > Stack Management > Rules, you can now enable the included Rules:

Go to Dashboards and select the Cribl tag to filter for all Cribl dashboards easily:

Go to the Cribl Metrics – Overview dashboard—a go-to hub for effortlessly gauging your cluster’s performance.

Quickly see if Groups, Workers, or Worker Processes stopped reporting metrics:

At a glance, spot outliers for workers in the Overview. Afraid of load distribution imbalance? The dashboard shows this too:

You can Drill In to the details of Worker Processes or Sources and Destinations:

After narrowing down a time range, we can open the Cribl Home – Logs dashboard, to see if the logs provide clarity:

What’s Next?

So now you’ve seen how quickly and effectively we can monitor Cribl. Clone and create your dashboards or use some of the features not mentioned here, such as Machine Learning or ES|QL.

Also, follow the instructions on our GitHub page for the whole experience. Feel free to contribute or report issues there.

You’re welcome to share feedback and ideas in our community slack channel. Are you not a member of our Slack Community? Join us!

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.