Historically, customers with SIEM/Security Analytics tools started with the likes of ArcSight or Nitro. As the amount of data grew and the variety of data sources increased, they faced the challenge of creating new connectors for each additional data source. This took plenty of time ( weeks at times ) and caused lots of risks as gaps appeared in the security posture while waiting for these new data sources.
Then came tools engineered to index any data source without needing custom connectors. Splunk and open-source tools like Logstash grew in popularity due to the ease of ingesting new data sources.
Customers started adding more data sources to meet their security analytics demands. And with the benefit of this new era of tools, it took less time to parse the new data sources. This made their security analysis tools grow into big data platforms. And now they were also routing application and infrastructure logs to these same platforms. They were able to get business intelligence and KPIs from the same log sources. And the number of users increased as the ROI started growing beyond security teams. Even the observability teams were getting on board.
These platforms then got bigger and required lots of tuning as the amount of data ingested grew. The hardware required to run at an optimum level where the user can get responses reasonably fast also got more expensive. And they had to start hiring dedicated resources just to maintain these platforms.
Then, they started using Cloud services from the respective vendors of these platforms. Initially, it worked very well. Performance was acceptable. With digital acceleration, the requirements for security and observability to have visibility into all systems grew even further. With that, the TCO for their Cloud vendors also went up.
Customers started looking at optimizing their logging by only routing the required fields to their cloud analytics tools. Cribl Stream became popular as it came with a user interface that allowed the engineers to create pipelines that helped remove unnecessary data prior to reaching its destination. And it came with a flexible API that allowed it to connect to multiple analytics tools (e.g. Azure Sentinel, Exabeam, Elastic, DataDog, Splunk, Grafana Cloud, NewRelic, Google SecOps (Chronicle))
Suddenly, customers were able to get better ROI by sending more optimized data through Cribl to their analytics tools. Given that data is growing exponentially, Cribl Stream was able to help customers get more with less. And with a larger cloud adoption from multiple cloud providers, Cribl was able to connect via APIs to the likes of Mulesoft, O365, Wiz, SailPoint and route the right data to the best destination.
Even customers who were using open-source tools like Logstash discovered that maintaining them in their environment was becoming more complex and expensive. We are seeing a growing number of customers who are now replacing their Logstash servers with Cribl, as they can eliminate the need for multiple hardware ( hundreds in large-scale environments where they ingest TBs) of data daily.
In one customer site with a large Elastic deployment, they were able to switch off up to a hundred Logstash servers by replacing them with a hybrid deployment of Cribl Stream, where they deployed the workers in both their Azure & AWS VPCs. This reduced their TCO significantly as they were able to :
Switch off the Logstash servers and save on compute costs from their cloud providers
Route more optimized data to their Analytics platform of choice
Reduce their egress costs as they needed security visibility from multiple cloud providers and by deploying the hybrid Cribl workers, they were limiting the amount of traffic leaving their VPC
Store unfiltered data in their respective VPC utilizing blob storage for compliance and security investigation requirements
Route optimized logs to their Elastic deployment from Cribl for their observability team
Routing to the Observability Team
As the teams were replacing the Logstash servers with Cribl, they realized that multiple log sources were being sent to both security and observability teams. This was a result of years of out-of-control data explosion. And they were being charged by both of their security and observability analytics tools for the same data. (Double-dipped)
That's when they realized that they could also use Cribl to decouple the logs and route security-centric logs to their SIEM solution and observability ( e.g., Application performance logs, and Debug logs) to their Observability solution. The savings were twofold. The performance improvement was significantly higher as they had removed the bottleneck of Logstash servers, which required constant tuning and upgrades.
Overall, they are switching off over a hundred Logstash servers in their environment. Even though these were open-source solutions, the savings in hardware ( Cloud VMs) and administration were a bonus, on top of improvements in data quality and anticipating data growth in coming years.
The ROI achieved here has been a big turning point for the customer, and now they are expanding their footprint with Cribl Cloud’s hybrid deployment. Give it a go yourself here!
