x
Blog-Header-Test

Get Swept Off Your Feet by Cribl Stream 4.5: Converting Dimensional Metrics to the OpenTelemetry Protocol Format with the OTLP Metrics Function

February 14, 2024
Written by
Desi Gavis-Hughson's Image

Desi Gavis-Hughson leads solutions marketing at Cribl. Prior to joining Cribl, Desi gaine... Read Mored over ten years of experience selling and marketing technology to IT and Ops leaders in commercial real estate, financial services, the media, and the public sector. Desi attended Princeton University, where she majored in East Asian Studies. Read Less

Josh Biggley's Image

Josh is a 25-year veteran of the tech industry who loves to talk about monitoring, observ... Read Moreability, OpenTelemetry, network telemetry, and all things nerdy. He has experience with Fortune 25 companies and pre-seed startups alike, across manufacturing, healthcare, government, and consulting verticals. Read Less

In the dynamic world of observability and analytics, everyone’s looking for smarter, more efficient, and interoperable ways to handle their data. That’s where Cribl steps in, bringing you an exciting update to our product lineup. We’re thrilled to introduce the OTLP Metrics Function to Cribl Stream 4.5! This Function converts metrics into the OpenTelemetry Protocol (OTLP) format with ease! Alongside support for dimensional metrics like StatsD Extended and CloudWatch Metrics, the OTLP Metrics Function also ensures your Prometheus metrics are OTLP-ready.

It’s like setting up a successful blind date between your data and the broad compatibility of OpenTelemetry (OTel). No awkward first meetings here; we’re talking love at first integration.

The best part, at least for your developers, is that you reduce the burden on your devs to update their apps to a new metrics model. Plus, you can use best-in-class OTel-compatible tooling. Who doesn’t like options?

This new Stream 4.5 capability maximizes the utility of your data, so it fits seamlessly into your existing systems while opening up new possibilities for scalability and choice in your org’s observability ecosystems.

Introduction

Monitoring and observability tools have come a long way, constantly evolving to meet the needs of complex, modern systems. Prometheus, an open-source monitoring and alerting toolkit, has been a favorite for many, thanks to its robust approach to collecting and organizing data from different sources. Prometheus’ design centers around a multi-dimensional data model and a flexible query language, making it a preferred choice for monitoring highly dynamic service-oriented architectures.

As our digital landscapes grow (Read: Moar cloud-native technologies and microservices), we need tools that not only collect data but also speak a common language, making it easier to understand and use this data across different platforms. That’s where OpenTelemetry (OTel) comes in.

Positioned under the Cloud Native Computing Foundation (CNCF), OpenTelemetry offers a comprehensive collection of APIs, libraries, agents, and instrumentation standards. These components work together to standardize the collection and exportation of telemetry data in a vendor-neutral format. OpenTelemetry’s initiative is a collaborative effort aimed at unifying observability frameworks, so that traces, metrics, and logs are consistently understood and processed across diverse environments.

The Problem with Prometheus

That said, pimpin’ merging data from Prometheus into the OpenTelemetry ecosystem ain’t easy. There are some fundamental differences in the data models, metric types, and aggregation methods at play. They’re a bit like two different dialects of the same language. Cribl’s new feature shines here as a bridge into the OpenTelemetry Protocol format. This way, you get the best of both worlds: the rich detail of your data and the broad compatibility of OTel.

Screenshot showing Prometheus metrics, before conversion with Cribl Stream 4.5.

By introducing a Function to seamlessly convert metrics like Prometheus into OTLP, Cribl is addressing a critical gap in observability.

The OTLP Metrics Function as seen in Cribl Stream 4.5.

When we simplify this process, we extend the utility of Prometheus within the OpenTelemetry framework without the need for complex Processors and Extensions. This way, Cribl enables you to unlock the full potential of your data, regardless of its source or format, positioning your organization at the cutting edge of technology and maintaining a competitive edge in an increasingly data-driven world.

Use Cases

Common Metric Theory/Consistent Vendor-neutral Methodology

OpenTelemetry Metrics semantics present a unified solution to the challenge of diverse, vendor-specific metric schemas within observability platforms. By standardizing telemetry data representation, you get a consistent, vendor-neutral methodology for collecting, querying, and interpreting metrics across various systems. That streamlines decision-making and troubleshooting for teams, fostering a cohesive environment free from vendor lock-in.

Vendor-agnostic Routing

Building on this foundation, the OpenTelemetry project advocates for a vendor-neutral telemetry framework that puts the generation, collection, management, and export of telemetry data first, deliberately separating these functions from storage and visualization to encourage a modular observability landscape. This architecture promotes a singular telemetry format as the source of truth, so organizations can select their tools without getting rekt by compatibility issues or hassled by reinstrumentation.

Enriching Metrics With Application Context

Taking it a step further, imagine linking every metric back to its origin within your application.

Nice, right?

OpenTelemetry’s Context Propagation and Baggage features allow for the sophisticated association of metrics with application context. Converting metrics from traditional formats, like Prometheus, into the OTLP format connects metrics with traces and logs, and wraps them in rich context through trace and span IDs and/or Resource attributes. That explicit and implicit contextual info means improved operational insights, reduced Mean Time To Resolution (MTTR), and better overall operational efficiency for your complex distributed environments.

Realize Integration Without Complex Configuration

The latest addition to Cribl Stream elegantly integrates dimensional metrics (e.g., Prometheus metrics) into the OTLP framework, smoothing over the complexities of observability integration. This simplification amplifies the value of Prometheus and other metrics within the OpenTelemetry ecosystem, removing the need for complex configurations.

Screenshot showing Prometheus metrics after conversion to OTLP format using Cribl Stream 4.5.

Wrapping Up

Let’s recap the benefits again for the people in the back.

With Stream 4.5, you unlock the ability to convert dimensional metrics to the OTLP format, so you can:

  • Bring uniformity to metric handling across platforms with OpenTelemetry (OTel) Metrics, ensuring consistency.
  • Simplify operations with a cohesive strategy that accelerates decision-making and troubleshooting for teams.
  • Encourage a flexible observability environment with OTel’s architecture, enabling seamless tool integration without compatibility issues.
  • Provide deeper insights by associating metrics with application specifics for a comprehensive understanding of operations.
  • Enable a clearer picture of system performance by fusing metrics with contextual details, especially in complex settings.

Ready to get started? Upgrade to Stream 4.5, and head to our documentation for a step-by-step walkthrough.


 

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.

.
Blog
Feature Image

Hello Vegas! Cribl @ AWS re:Invent 2024

Read More
.
Blog
Feature Image

Simplify OpenTelemetry Metrics with Cribl Edge OTLP Conversion

Read More
.
Blog
Feature Image

Cribl and CrowdStrike Partner to Transform Data Management for SIEM Solutions

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?