Every enterprise collects and stores massive amounts of security and observability data but struggles to get value outside of operations and security teams. These datasets can offer enormous value to business operations and enterprise reporting teams if they have access to the data in their toolsets. BizOps needs to optimize batch planning and the enterprise reporting teams need to reconcile how many assets the enterprise owns versus the number it has under support contracts. Both teams need the correct data to produce accurate results. How can enterprises extract full business value from its security and observability data while limiting cost and complexity?

TL;DR

• Security and Observability data offer business value across the enterprise to data analytics, business operations, and reporting teams.
• Cribl Stream makes it easy to collect, normalize and ship data to cloud data warehouses like Snowflake.
• Snowflake’s data ingestion service, Snowpipe, automates loading data from your object-store.

Data siloing is an old problem. Either data is only available to one team or business unit or no one knows it is there in the first place. Cribl Steam unlocks the value of all of your observability data. Operations and security teams can easily route their data to enterprise analytics and reporting tools to provide executives with on-demand data for conversation rates, website traffic, and a host of other business metrics. No more waiting on end-of-day loads into the data warehouse. Get the business data that matters now. Cribl Stream not only has a major impact on security and observability teams, but also for executives looking for timely, better data to improve decision making.

Campaign Management – Speed Wins

A company wants to make targeted offers to consumers that use its kiosks. For example, an unmarried person 25-35 with a specific profile gets one offer and a married person at the same age gets another offer, and so on. This is a very common pattern to offer each consumer what they want to hopefully get the best offer conversation and generate more sales. But what if your assumptions are wrong or a competitor is running a campaign at the same time and your conversation rate is bad? How do you know you have a problem? Do you have enough time to do something about it?

Traditionally your application will feed data into your database and then your DBAs will load data into your business analytics solutions overnight to minimize load on your customer-facing databases. This drives end-of-day/end-of-week reporting to your analysts to reveal that conversation rates are poor.

That is too much time to get feedback from your campaign. Users are seeing bad offers that generate at best indifferent consumer perception and your company is losing money from failed conversations. How do you speed up the feedback loop to get this information to your analyst faster?

Operations Analytics with Cribl Stream and Snowflake

The solution is to leverage your observability-focused application logs to feed ad conversion data into your data warehouse to drive on-demand analytics across your enterprise. Your ad campaign team can know that conversion rates are poor with hourly reporting and be able to take action right away instead of waiting for the next day’s reporting. The team can adjust the campaign and watch conversion rates improve all in the same day. This capability breaks down traditional silos to get business value from long-ignored data sources. Time really is money.

Below are basic steps to build a Cribl pipeline to support pushing data to Snowflake over SnowPipe. I am assuming you already have your logs flowing through Cribl Stream. It is super helpful that you can use your existing log shippers with Stream.

First, build a pipeline to transform your data to make it useable in Snowflake

Transform your fields to the right format

Apply a parser to the data

Rename fields to fit with data warehouse scheme requirements

Drop data you do not need

Set up your S3 bucket to message SnowPipe when data is ready for loading

As you drop data in your bucket SnowPipe will automatically consume it into SnowFlake. SnowFlake has documented the S3 bucket process here.

This is a great example of how a basic workflow can extract serious business value from observability data.

Bottom Line on Cribl Stream and Snowflake

Cribl Stream and Snowflake enable enterprises to extract significant business value from traditionally non-business focus data. Unlock the value of your security and observability data with Cribl Stream.

Try Cribl’s free, hosted Stream Sandbox. I’d love to hear your feedback; after you run through the sandbox, connect with me on LinkedIn, or join our community Slack and let’s talk about your experience!