Simplify OpenTelemetry Metrics with Cribl Edge OTLP Conversion

December 2, 2024
Written by
Raanan Dagan's Image

During his many years at Cribl, Splunk, Cloudera, and Oracle he was part of multiple impl... Read Moreementations of security, analytics, cloud, open-source, and IT use cases as well as big data and data lake projects in complex environments. Raanan is a global resource with 30 years of experience building large data clusters. He has helped thousands of customers, including some who ingest several hundred terabytes per day and store multiple petabytes of data. Read Less

Categories: Cribl Edge, Engineering

Cribl Edge can send data to OpenTelemetry in several different ways. In this blog post, we’ll focus on the OpenTelemetry Metrics. In the blog, we’ll talk about Cribl Edge, but what we say applies to Cribl Stream, too! We will cover how to use Cribl Edge to collect Linux System Metrics, transform them into the OTLP Metrics format, and deliver them to an OTLP Destination.

Why Collect OpenTelemetry Metrics?

The goals of collecting OpenTelemetry Metrics are centered around a few key objectives. First, it aims to enable the connection between metrics and other signals. For instance, this allows for the correlation of metrics and traces, providing richer insights. Additionally, OpenTelemetry aims to support existing metrics instrumentation protocols and standards. At a minimum, this includes full support for Prometheus and StatsD, ensuring that users can leverage OpenTelemetry clients and the Collector to collect and export metrics while maintaining the same level of functionality as their native clients.

Cribl Edge can help automate this conversion process from normal metrics to OpenTelemetry metrics.

Using Cribl Edge to Collect Metrics

Cribl Edge supports many options for collecting metrics, including Linux, Windows, Kubernetes, and more.

In this blog, I am using Cribl Edge with Linux System Metrics, for example, collecting the CPU metrics usage on the node.

 

A screenshot of a computer Description automatically generated

Transforming Metrics Events into the OpenTelemetry Protocol Metrics Format

Adding a pipeline or a pack with the Cribl OTLP Metrics function will transform dimensional metrics events into the OpenTelemetry Protocol metrics format.

A screenshot of a computer Description automatically generated

The OTLP Metrics function default behavior can take the live data and the original metrics event, for example, Idle CPU metrics, and transform it.

Before –

A screenshot of a computer code Description automatically generated

After –

A screenshot of a computer Description automatically generated

Also, Cribl OTLP Metrics supports the option to do batching.

Enabling the batching of OTLP Metrics by shared top-level `resource` attributes will combine many metrics attributes to fewer events.

A screenshot of a computer Description automatically generated

Sending the Open Telemetry Protocol Metrics Format to the OTLP Metrics Destination

You can use Cribl Edge to route these newly transformed metrics events to any destination of your choice.

A screenshot of a computer Description automatically generated

Cribl Edge streamlines the collection, transformation, and routing of OpenTelemetry Metrics. Whether you’re working with Linux System Metrics or data from environments like Windows or Kubernetes, Cribl Edge makes it easy to convert metrics into OTLP format and send them anywhere. If you’re looking for a scalable and efficient way to manage OTel data—or any telemetry data—Cribl Edge has you covered.


 

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.

 

.
Blog
Feature Image

Cribl: Empowering Data Freedom with Open Standards and Unmatched Flexibility

Read More
.
Blog
Feature Image

Hello Vegas! Cribl @ AWS re:Invent 2024

Read More
.
Blog
Feature Image

Cribl and CrowdStrike Partner to Transform Data Management for SIEM Solutions

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?