A Place for Everything and Everything in Its Place

Written by Kevin Morris

May 23, 2023

With Cribl Stream, our customers are experiencing choice and control over their data that would have been a pipe dream (or maybe I should say a pipeline dream) before. The ability to get the right data to the right destination in the right format is extremely powerful. Stream can optimize the data being sent to expensive destinations; you can remove unnecessary or redundant fields, drop unnecessary events, or even pull valuable metrics from verbose logs.

Optimizing your data has a few benefits:

Just In Case

There is another option for data optimization that is often overlooked. Many organizations have data sources that are being sent to a SIEM / System of Analysis. I mean, that’s where we send our data, right? Some of these sources may need to be captured and stored for compliance reasons but rarely get searched, if at all. So, why not put that data in a more appropriate place? How about routing that data to inexpensive object storage such as Amazon S3 or a Data Lake? And how about storing it in a vendor-neutral format? After all, if we’re not really searching the data, why do we need it in some powerful analytics tool?

I know what you’re thinking: “But what if I need to search for something in that data?” I don’t have the power to read minds or anything, it’s just that this is usually the next question. Well, Stream has you covered. If, and only if, you should need it, Stream can replay that data from those more inexpensive storage options. That data can be routed and processed however it needs to be and sent off to whatever destination or destinations it needs to go to. And remember, we can optimize that data on the way there!

Take Control of Your Observability, Security, and Telemetry Data - Cribl

Just Looking

If you are a Cribl Cloud customer, you have an even more powerful option than replaying the data back into Stream. Cribl Search allows you to search data in place, without needing to send it to a SIEM or system of analysis first. If you are sending that rarely searched data to Amazon S3 as an example, you can utilize Cribl Search to search your data in that Amazon S3 bucket. Once you have narrowed the search down to the right data, you can send it off to Cribl Stream for routing, transformation, tool optimization, etc.….

A screenshot of a computer Description automatically generated with medium confidence

Conclusion on Optimizing Your Observability Data

If this sounds interesting to you and you want to learn more about Cribl Stream, check out the following links:

Questions about our technology? We’d love to chat with you.

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?