Telemetry volume and costs continue to be a growing problem, even for large organizations with the resources to tackle it.
Like many companies out there, our telemetry volume continues to increase. It’s expensive, unwieldy, and hard to weed through to identify what's really important. We’re up to a few hundred TB/day in terms of logs generated — but with Cribl, we’re able to intelligently curate the data that we’re sending to our monitoring tools, helping us to save effort and budget..”
Senior Manager
SRT Group
The Site Reliability Tools (SRT) team is leading the project, with Cribl Stream at the center of a unification effort to bring their 1400 microservices into one platform.
“Our new platform comes with agent-based instrumentation and an automatic flow of logs through Cribl Stream. As the services adopt it, they can do all kinds of ETL activities with that data and easily get it to its final destination.”
Senior Manager
SRT Group
Making strategic changes for long-term efficiency
The changes have had an immediate impact, and the organization foresees the benefits continuing far into the future. The initial driver for bringing on Cribl was cost optimization, and now they’re also seeing improvements in managing infrastructure.
“We tackled the logging space first because that was the biggest offender, but moving all that data around can be really complex from a plumbing perspective. Cribl gives us centralized location that really helps us manage and route that telemetry to where it needs to go.”
Senior Manager
SRT Group
But logs are also just one piece of the puzzle — now that they can easily filter and sample them on their way to Splunk, they can move on to managing the rest of their complex network. Once the rest of their telemetry data is where it needs to be, visualization tools become much more powerful.
“In the long term, we’re looking to bring telemetry as a service to our customers as well — Cribl is going to play a pivotal role in that. As we get metrics and traces flowing through this common pipeline, our game developers will be able to tap into that telemetry and use it as they see fit.”
Senior Manager
SRT Group
Managing Infrastructure with the team in mind
Changing instrumentation is a big lift, so the SRT group is making as many of these infrastructure changes as they can without impacting the service teams.
“Making adjustments in our tool sets while minimizing the ask for our service teams is really critical to us. Cribl lets us test and swap out different tools while keeping our monitoring and visualizations the same — the uninterrupted plug and play makes a huge difference.”
Senior Manager
SRT Group
As they build with choice and control for the future in mind, the team has also started to think about instrumenting with OpenTelemetry instead of any specific vendor’s agent. No matter what tools or formats they decide to go with, Cribl gives them the flexibility to migrate and integrate efficiently.
“Cribl being vendor-agnostic makes it a lot easier to pull in a new tool, kick the tires on it, and migrate to it piece by piece. Since we can be selective about what we're pushing over to the new tool, we get a lot more control and flexibility.”
Senior Manager
SRT Group
Building for the future
As the organization moves toward a multi-tenant model, one of their goals is to provide observability as a service for more of the organization.
“The self-service capabilities and Cribl are going to give our stakeholders transparency around costs, more insight to what their services are emitting, and the ability to pull the necessary levers to increase efficiency.”
Senior Manager
SRT Group
They’re also considering how to bring other tools like Cribl Search into the mix.
“We’re already seeing the benefits of using Cribl Stream to do the transformations, routing, and filtering — and to replay data from S3 into Splunk. But if we can use Cribl Search to actually search data in place, we imagine that will give us huge time savings–and an easier path to accessing archived data.”
Senior Manager
SRT Group
Find out more about how you can use Cribl, the Data Engine for IT and Security, to empower your organization to transform your data strategy.
Get Cribl, and take control of your data.
TL;DR
Cribl Stream acts as the primary telemetry pipeline solution to manage logging and will serve as the data engine for an entire telemetry platform as a service offering for internal developers.
SRT team is using Cribl Stream to unify their 1400 microservices into a single platform, with agent-based instrumentation and automatic flow of logs.
Cribl will play a pivotal role in managing and routing telemetry data as a service.
Cribl's vendor-agnostic approach allows the company to test and swap out different tools while keeping their monitoring and visualizations consistent.