Goats on the Road: Getting More Value From Observability Data

August 23, 2022

The best part of my job is talking with prospects and customers about their logging and data practices while explaining how Cribl focuses on getting more value from observability data. I love to talk about everything they are doing and hope to accomplish so I can get a sense of the end state. That is vital to developing solutions that provide overall value across the enterprise and not just a narrow tactical win with limited impact.

I spent last week meeting with prospects and customers in the midwest. Chicago BBQ is underrated, and Green Street Smoked Meats is a favorite. Over brisket and beer, we discussed the challenges of getting value from observability and security data with the leadership from a Chicago financial powerhouse. They fully recognized it needed to change how data was collected and processed to support IT and Security use cases but were very concerned about the time, effort, and risk of making changes.

Good leaders must consider displacement’s cost and effort even with the best technology options. If the existing technology is just good enough, why would you spend the time and money to make any change? Sometimes mediocre tech is good enough even if it is a pain in you know what to manage.

Risk is another variable that is not widely considered as well. Good leaders ask if the migration to a better platform will compromise our existing IT or security posture and weaken us, at least in the short term.

Finally, the economy is on everyone’s mind. Companies are preparing for a possible downturn by proactively cutting spending. Even companies that are ok with their current state are reaching out to understand options if they need to optimize costs.

It was fantastic to see the team had already worked past the idea of the displacement costs being too much to consider Cribl products. Cribl tools are designed to minimize displacement costs so you can place Cribl Stream between your sources and destinations. Use your existing agents and data to get more value from the data analytics platforms you already have. Cribl does not require you to change out your agents or data platforms. The team was looking for far more than just lowering license costs and was very focused on the following:

Scaling Staffing

Scaling staffing is a big concern. Hiring and retention are massive issues for companies and they are actively looking for ways to automate work to help existing teams get more work done with less effort. They know they cannot hire more skilled people and have to put effort into supporting existing teams. Tools like Cribl Stream and Edge automate so many of these tasks so teams can get beyond managing data to support advanced analytics, which is they can deliver business value.

Decoupling Dependencies

Decoupling dependencies is a new idea that I am glad to see spreading rapidly. The idea is to separate the vendor/method for gathering data from how you analyze and store data. This gives you options for using multiple data analytics vendors instead of being locked into one vendor who also supplied the method to gather data. Another big trend is storing a full-fidelity copy of data in the customer’s own object storage with the idea of preserving options for how that data can be used in the future and lowering retention costs. Users finally understand that once you put your data into your vendor’s cloud, that data is no longer used, and you lose options for getting value from your own data. In addition, long-term retention costs can be 2-3x higher than using your own storage.

Cribl Stream makes it easy to pull data from your object storage and push it to data analytics providers like SnowFlake or create an ad-hoc threat hunt data lake to search with Humio or whatever makes sense at the moment. The idea is to preserve options and give your organization the choice and control to make decisions that meet your needs without considering whether my vendor will easily support what I want to do.

Getting More Value from Observability Data

More and more enterprises are starting to understand their observability and security data holds much more value than basic observability and security use cases. This data can be mined for all sorts of insights to support marketing, insider threat, cyber fraud, and traditional business analytics use cases. The potential value is enormous, and Cribl’s tools support giving teams options to fully unlock value from this enormous data pool. This is the sort of new capability that is attracting attention even in tough economic times.

I also had a design and architecture review with a brilliant engineer who asked the right questions about all the topics we had just discussed. It was fun to guide him through the higher level of thinking about how to structure your architecture to support current needs to decouple dependencies and be flexible enough to solve future problems. It was fun to see the growth from thinking like an engineer to thinking like an architect. Seeing the lightbulb of understanding is always a great moment. Working with people is fun, super rewarding, and a significant part of this job.

Meeting after meeting, I heard leaders tell me they want to reduce/remove vendor lock-in to push data to the right tool for the task and get beyond restrictions from existing vendors. If I am a dominant vendor in the observability/security space, I check in with my customers to ensure my relationship is solid. Leaders want options more than I have ever seen before and are unhappy about still feeling like their choices are limited.

Bottom Line

It was great to spend time with so many impressive teams looking at getting more value from observability data. We met engineers and architects who know their challenges and are ready to innovate with Cribl tools. I look forward to working with the teams to help solve real-world issues and free up time to help innovate their security and observability practices. I cannot wait to see the new ideas they will share with the Cribl community.

Try Cribl’s free, hosted Stream Sandbox. I’d love to hear your feedback; after you run through the sandbox, connect with me on LinkedIn or join our Community Slack and let’s talk about your experience!

.
Blog
Feature Image

Cribl Stream: Up To 47x More Efficient vs OpenTelemetry Collector

Read More
.
Blog
Feature Image

12 Ways We Sleighed Innovation This Year

Read More
.
Blog
Feature Image

Scaling Observability on a Budget with Cribl for State, Local, and Education

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?