Cribl Search is agnostic, allowing administrators to now query Snowflake datasets as they can dozens of other Lakes, Stores, Systems & Platforms.
The data that IT and security teams rely on to monitor network operations continues to grow at a 28% CAGR, and it’s stressing many organizations’ ability to analyze all this data effectively. In fact, in some cases, less than 2% of it ever gets looked at. This means valuable insights often get missed, and potential security threats go undetected, increasing the risk of breaches and operational inefficiencies. Critical datasets are routed directly to systems of analysis, while the rest end up in various data archives — out of sight but never out of mind.
As a result, data storage has become a big business, with multitudes of data vendors and cloud storage providers offering various storage types and pricing options, such as databases, data stores, object stores, and data warehouses.
One of these data warehouses is Snowflake, which enables data storage, processing, and analytic solutions that are faster, easier to use, and far more flexible than traditional offerings. It sounds like a service most organizations would be interested in, and based on their success, I would agree. But truth be told, there’s no best way or best place to store your data. For instance, customer transaction data might be stored in Snowflake, while log data sits in AWS S3, and security event data is kept in an on-prem SIEM. This creates challenges in correlating and analyzing data across these silos. As a result, no matter how good a service is offered, like with Snowflake, many organizations end up with key datasets distributed across multiple systems. This siloed data creates barriers due to inconsistencies in how it is stored, viewed, and retrieved — making it very difficult for administrators to get a holistic view of company data. Which is necessary to make critical business decisions.
But there is a relatively easy fix to resolve this, Cribl Search.
Cribl Search has added Snowflake to its list of dataset providers, providing administrators with another choice in where and how they store their data.
With Cribl Search, admins can now federate Cribl’s search-in-place capability to anywhere the data is located including in Snowflake’s Data Cloud. Unlike traditional federated search, which often involves copying or moving data to a central location before analyzing it, Cribl Search’s search-in-place allows queries to be executed directly on the source data. This minimizes data movement, reduces latency, and ensures that the most up-to-date data is being analyzed. Additionally (yes, there’s more), it helps cut storage costs and the potential for data sprawl since data isn’t duplicated across multiple systems. This provides a never-before capability that can simultaneously search and analyze data wherever it is located – from debug logs on a host, to archived data in cloud storage, even search data warehouses with our new addition of Snowflake as a query target.
Have a Snowflake account and want to take advantage of querying data in place and aggregating data from multiple sources (federated search) to optimize the data distributed to your IT and security teams? Great, Cribl can get you set up in a few easy steps:
First, create a Snowflake Dataset Provider – this tells Cribl Search where to look
Provide a unique identifier for the dataset provider to name it
Add a description
Enter your account credentials
Snowflake Account identifier & Username
Snowflake Private key & Private key passphrase
Next, create a Dataset – this defines what data to search within Snowflake
Give it a name and description (optional)
Provide information required to target the data
Warehouse name – defaults to the warehouse property of the user.
Database name- defaults to namespace of the Snowflake user.
Snowflake schema that contains the tables you plan to query
Name of the table, view or a query – Simple name or SQL query
No kidding, it’s really that easy; see below.
If you want to check it out with your data, click here. You’ll have a FREE Cribl account in 2 minutes and be searching your Snowflake account in 10!
If you want full configuration information, please check out our docs.
Cribl Search Capabilities
Ok, so we can easily query your Snowflake account, but what else can a single Cribl Search query access?
Data Lakes – including Amazon Security Lake, Amazon S3 and compatible
Object Stores – including the likes of Amazon S3, Azure Blob Storage, Google Cloud Storage and more
Analytics Services and platforms, like Azure Data Explorer, Elasticsearch, Opensearch, and Prometheus.
API Endpoints – including Azure, AWS, Google Workspace, Okta, Zoom, and even a Generic HTTP API option
And lest we forget integration with Cribl Products.
Cribl Stream, allowing you to shape, format and route data to the destination of your choice
Cribl Edge, offering the ability to query logs, metrics, and application data from edge nodes.
Cribl Lake, our lake solution aimed at low cost, long-term, full-fidelity data storage for your data.
What could you do with a Search-in-Place tool that simultaneously performs surgical queries at multiple locations, then shapes, filters, and aggregates the results before forwarding to your destination(s) of choice, be it SIEMs, log analysis tools, cloud storage, etc.
Wrap up
Search is designed for the unique requirements of IT and security data, allowing administrators to easily access and explore almost any system, storage type or API endpoint from a unified interface using a single, intuitive query language. This eliminates the need for multiple proprietary tools allowing you to access almost any data, storage, application, or vendor-specific systems. Then, surgically locate just the specific dataset required and route that data to your existing analysis system for deeper analysis.
Already have a Cribl account? Great! Try it out. No account? No worries, just create a free account here, it takes less than a minute, and you can immediately start searching your Snowflake data.
