Highlights
Reduced data onboarding times from months to days and, in some cases, hours
Filtered out unnecessary log data
Stored and accessed compliance data with Cribl Search
As an Enterprise Software Company focused on workload and device management underwent aggressive scaling, the Platform Provider’s security team saw their Splunk license spiral from 1.2 TB to 30TB a day in four years, causing serious log ingestion issues. To serve their 26,000 customers, seven of which are among the top 10 Fortune 500 companies, they needed to find a solution.
To deal with increasing log data from global AWS CloudTrail, EDR tools, and application logs that were breaking log streams, compounding event delays, and overwhelming forwarder infrastructure, the Software Company’s engineering team built DIY log pipelines.
While these efforts were initially successful, onboarding data was still a slow process. It soon became impossible for the small team to maintain the pipeline, and costs spiraled out of control. At this point, they knew they needed Cribl Stream.
“We realized that going down the homegrown path, we were spending a lot of money but not getting enough value. Cribl had the rich feature set and support capabilities we needed. We had a team at Cribl that allowed us to focus that time and effort back on other priorities within the business.”
Program Lead, Advanced Security Analytics
Streamlining data onboarding
The Software Company's biggest goal with Cribl was to onboard and manage data efficiently. Prior to using Cribl, the data onboarding process typically took months to complete. With Cribl Stream, the team were able to reach out to business owners and request that they drop their data in an S3 bucket and, from there, set up a pipeline.
“Cribl has really helped us streamline our interactions with the business. We're able to now just push out forms to them to say, ‘Hey, we need to collect your data and this is where we need this data to go. Here are your options.’ This allows them to operate on their own time and on their own terms to get us that data.”
Program Lead, Advanced Security Analytics
They could then review the data in Cribl, make the necessary changes and adjustments, and route the data where it needed to go, whether that was their SIEM or a secondary logging solution. This process reduced the time to onboard data from months to days or, for simpler datasets, hours.
“Before we had Cribl, I hated the data onboarding process. It was a nightmare. Everybody on the team hated it because it took so long. We like to be able to close things out and move on to the next thing as efficiently as possible. Cribl has allowed us to do that.”
Program Lead, Advanced Security Analytics
Data pipeline optimization
However, the team found that the onboarded data was messy, and many data sources didn’t allow for a one-size-fits-all approach. So, using Cribl Stream, they added guardrail conditions to add a field named “Wrong Format” and set it to true. They then set up a daily report in their SIEM to spit out “Wrong Format” datasets and fine-tune them to a usable format.
“Cribl allowed us to answer questions like: how useful are these logs? Are we bringing in the things that we need, or are we bringing in too much data? Is it too little data? Is it junk data? Did we check the box to say that we collected this data, but then when we go and look at it, we realize it contains no useful information?”
Program Lead, Advanced Security Analytics
From there, the team needed to determine what data was security-relevant, so they worked with the security team to get their perspective. This allowed them to create a data hierarchy, using Cribl to ship only the most high-value data back into the SIEM and dump unnecessary data.
“Being able to collect security data is important because if we can’t, we can’t build detections and we can't secure an environment. Cribl allows us to tell business units that we’re collecting and monitoring their data and that if there is a problem, we can resolve it in a reasonable amount of time.”
Program Lead, Advanced Security Analytics
Storing and accessing compliance data
The Software Company also used Cribl to route compliance and supplemental data to a S3 bucket. With Cribl Search, they can easily find this data when they need to demonstrate compliance or inform investigation efforts.
“Without Cribl, we wouldn’t be able to meet the compliance requirements our customers demand. If we can’t capture and access compliance data, we can’t pass audits, so if we don’t meet these requirements, someone else will, and we’ll lose customers.”
Program Lead, Advanced Security Analytics
Exceptional customer support
Throughout this process, the Software Company benefitted from Cribl’s customer-first approach. They worked closely with Cribl over Slack, sought advice from the community, and took advantage of learning resources. One of the biggest benefits was free training.
“Training is free. I think that says a lot about a company and its product and how much it stands by it. Cribl's got some great training and certification paths, and it's all free.”
Program Lead, Advanced Security Analytics
TL;DR
An Enterprise Software Company replaced their DIY data pipeline to:
Reduce data onboarding times from months to days or even hours
Filter out unnecessary log data and focus on security-relevant data
Store and access compliance data with Cribl Stream and Search
Benefit from Cribl's exceptional customer support, including free training