Last week, Cribl launched the latest component of its observability architecture: Cribl Edge. ICYMI, Cribl Edge is a next generation observability data collector that greatly simplifies gathering your metrics, events, and logs. Edge incorporates all of the capabilities of Cribl Stream’s workers, allowing you to route, redact, filter, and enrich data directly from the source. Why is this important? On our webinar announcing Edge (now available on-demand), we asked well over a hundred people how they felt about their observability data spend, the number of agents deployed, and if they were using APM. Some of the responses were what you’d expect, while others were surprising.

Let’s get to the data!

First Up, How’s That Spend Treating You?

The results are about what you’d expect. According to figure 1, over 80% of people feel they’re paying too much for all those logs, metrics, and events. I’ll argue those people don’t feel they’re paying too much but that they’re getting too little value for what they’re paying. If they could choose what data they kept and where they could send it, I’d wager they’d feel differently. Unfortunately, the reality is they can’t because they’re locked into punitive pricing models built on data silos.

Figure 1. Do you think your current observability data bill is too high, too low, or just right?

By pushing Cribl Stream capabilities to the Edge, our customers have more choice about not just what they do with their data but where they do it. (Oddly, 3% of people believe they’re paying too little. I’m sure other vendors will be in touch with them.)

Agents Abound

Beyond controlling costs and increasing the value of o11y data, there’s a massive challenge simply managing the number of agents required in a modern environment. Nearly 60% of our webinar respondents have deployed three or more agents, with 34% deploying over five (figure 2). Now, you’re likely thinking, “That’s not so bad. What’s the big deal? Set up your agents and you’re done, right?”

Wrong. Agents require upkeep in the form of upgrades, configuration changes, and troubleshooting when your configuration changes don’t work. Not to mention the number of instances you’re deploying, which frequently numbers in the high tens of thousands or even hundreds of thousands of instances. Your five-plus agents is really half a million agents, maybe more. Remember that scene in The Matrix Reloaded when the Agent Smith clones pour out of the building to attack our intrepid protagonist? It’s like that—every day.

Figure 2. How many different agents do you currently have deployed to collect data?

Cribl Edge takes a first-principles approach to manage these massive fleets of agents. First, Edge automatically discovers logs on the deployed system. This means there’s no need to manually configure Edge when you initially deploy it or to reconfigure it if an application change moves a log file. Next, entire fleets of Edge instances can be upgraded directly from the web-based control plane. And because you’re getting the same capabilities as Cribl Stream, you also get the UX, including preview capabilities for your reduction, enrichment, filtering, and redaction use cases.

Everybody’s Monitoring Applications (Kinda)

Finally, we asked our audience about APM usage (figure 3). The numbers were about what you’d expect, with 58% of respondents using APM systems. These include products like New Relic, Dynatrace, and DataDog. My guess here is even with 58% penetration, APM usage isn’t as high as those respondents would like. By our estimates, APM is only used on about 10% of enterprise applications, often the most mission-critical, due to cost constraints. It can be prohibitively expensive to run APM on everything you’d like, and you still may not get the information operations and security teams need.

Figure 3. Are you currently using any Application Performance Monitoring (APM) system?

We’re addressing this monitoring gap by including AppScope with Cribl Edge. If you’re unfamiliar with AppScope, it’s our open source, black-box instrumentation utility. By combining Edge and AppScope, you can teleport to a remote instance, turn on dynamic instrumentation for any process, and process that data with Edge or Stream. Then, when you’ve collected what you need, you can turn off instrumentation. It’s a remarkable innovation for operations and security teams.

If you want to learn more about Cribl Edge, you can always check out the sandbox. Make sure to join our community if you have questions, or just want to talk about goats.

(Photo by Casey Horner on Unsplash)