With Cribl Stream, our customers are experiencing choice and control over their data that would have been a pipe dream (or maybe I should say a pipeline dream) before. The ability to get the right data to the right destination in the right format is extremely powerful. Stream can optimize the data being sent to expensive destinations; you can remove unnecessary or redundant fields, drop unnecessary events, or even pull valuable metrics from verbose logs.

Optimizing your data has a few benefits:

• Make room for additional data sources that could provide much-needed visibility.
• Prevent or limit costly license expansions with your SIEM / System of Analysis vendor.
• Improve the performance of those downstream systems.
• Improve the value of the data you are sending. Less hay, more needles so to speak.

Just In Case

There is another option for data optimization that is often overlooked. Many organizations have data sources that are being sent to a SIEM / System of Analysis. I mean, that’s where we send our data, right? Some of these sources may need to be captured and stored for compliance reasons but rarely get searched, if at all. So, why not put that data in a more appropriate place? How about routing that data to inexpensive object storage such as Amazon S3 or a Data Lake? And how about storing it in a vendor-neutral format? After all, if we’re not really searching the data, why do we need it in some powerful analytics tool?

I know what you’re thinking: “But what if I need to search for something in that data?” I don’t have the power to read minds or anything, it’s just that this is usually the next question. Well, Stream has you covered. If, and only if, you should need it, Stream can replay that data from those more inexpensive storage options. That data can be routed and processed however it needs to be and sent off to whatever destination or destinations it needs to go to. And remember, we can optimize that data on the way there!

Just Looking

If you are a Cribl Cloud customer, you have an even more powerful option than replaying the data back into Stream. Cribl Search allows you to search data in place, without needing to send it to a SIEM or system of analysis first. If you are sending that rarely searched data to Amazon S3 as an example, you can utilize Cribl Search to search your data in that Amazon S3 bucket. Once you have narrowed the search down to the right data, you can send it off to Cribl Stream for routing, transformation, tool optimization, etc.….

Conclusion on Optimizing Your Observability Data

If this sounds interesting to you and you want to learn more about Cribl Stream, check out the following links:

• Sign up for a cloud instance on Cribl.Cloud (FREE up to 1TB/day).
• Try out our Cribl Sandboxes.
• Get free training and certifications at Cribl University.
• Join the Cribl Community or ask a question on our Q&A Forum: Cribl Curious.