At Black Hat this year, I had many versions of the same conversation. Security leaders from hospitality, financial services, insurance, and enterprise SaaS shared a common story. Telemetry volumes are exploding while budgets grow only slightly. Everyone is trying to manage their SIEMs by migrating, adding new tools, or even considering a SIEM-less approach.
Then I'd ask where they are going to put their data. Almost every time the answer was "we're going to land it in the lake."
And "the lake" almost always meant Snowflake or Databricks.
The good faith part
Snowflake and Databricks are excellent at their intended functions. They are great platforms for BI, ML training, and long-horizon analytics across curated business data. That is the job they were designed for and they smash it.
Security telemetry is a different challenge. I have built several security data lakes on platforms like RedShift, Google Data Lake/BigQuery, and Snowflake. They were all extremely hard to implement, required loads of translations into data engineering parlance from security vernacular, and I kept running up against the same challenges.
The cost nobody puts in the business case
The hidden failure point for these projects lives outside your budget spreadsheet. Enterprise data teams prioritize high-visibility projects like revenue analytics and board-mandated executive dashboards because those initiatives have direct C-suite & Board level backing.
Security telemetry pipelines inevitably land on the wrong side of these massive engineering backlogs. Constructing a security lake on a shared warehouse forces your team to wait months for an initial sprint. These prolonged wait times exist for critical schema updates when log formats change.
Effective detection engineering relies on immediate access to raw data and the flexibility to pivot during active investigations. Standard warehouse workflows introduce bureaucratic bottlenecks that prevent security teams from moving at the speed of a live threat.
What owning your own lane looks like
Security data needs its own path. Forcing IT and security telemetry into a general-purpose warehouse creates a conflict of priorities, resulting in bottlenecks and high costs. Cribl Lake, for example, is different because it is purpose-built for the unique demands of observability data.
While warehouse-based lakes require rigid schema definitions and constant maintenance, Cribl Lake simplifies this with automated schema handling for Parquet—no upfront planning required. It treats security telemetry as a first-class citizen, supporting logs, metrics, and traces natively. You maintain full ownership of your data in open formats, eliminating vendor lock-in, and you have the flexibility to use either Cribl-managed storage or your own BYOS infrastructure.
This approach changes the economics of long-term retention. Because you can land high-fidelity data in low-cost storage, you can keep what you need for incident response without the high query costs of a traditional warehouse. When you need that data, it is immediately available for search and replay without the delay of rehydration. It brings managed simplicity to the security stack, letting your team focus on detections rather than managing infrastructure.
This is the foundation for the AI-ready security operations center. Autonomous agents can only be as effective as the data they can reach, and Cribl Lake ensures that data is accessible, affordable, and under your control. Start with the data, and stop building your security strategy around someone else's backlog.
