We’re delighted to officially announce today the general availability of Cribl Stream!

Cribl Stream delivers unique intelligence, control and compliance over your logs and metrics data in real-time. It puts the admins in control and gives users the right data, with the right context, delivered to the right systems to enable operations, security and analytics without pushing every requirement back to developers, vendors or source systems. Cribl Stream is purpose-built for real-time and enables enterprises to collect 100% of data that might be interesting and determine at ingestion time what is valuable. Users can now secure, enrich, and route operational data in real-time to maximize the value and meet the needs of their business.  Various sources and destinations are supported but as the market leader, Splunk was selected as the first system optimized by Cribl.

Get Cribl Stream Now!

Beta Program

We couldn’t be more thrilled at the amount of participation we received in our beta program. We worked with dozens of customers and partners in the Splunk ecosystem. We uncovered exciting new use cases, like using sentiment analysis to discover real time security threats to the mundane of simply helping people easily shuffle off some traffic to a test environment. From one of our users:

“Overall I couldn’t have been more impressed with your product – it really is the perfect companion tool to managing the data that comes into Splunk. It singlehandedly addressed a dozen or more items on our wish list of things that you want Splunk to do to make life easier. “

Our GA release simply would not be possible without the amazing feedback we received from our beta participants. We developed a number of new capabilities out of use cases and requirements we received from our participants, and as always, thanks for the bug reports!

Use Cases

Cribl has a number of great use cases, many of which we discovered through our beta program.

• Noise Reduction & Data Shaping
  • Cribl helps you minimize junk and noise from logs and other sources. For high volume, low value data sources like web access logs, firewall & router logs, VPC/Netflow flows, smart sampling allows you to keep interesting events like errors or rejects/denies while sampling more voluminous but less interesting events like successes or accepts/allows . In addition, often times operational and security data is not in good shape for its eventual analytics destination. With Cribl you can transform, parse and re-format logs and events of almost any shape.

• Ingestion-time Enrichment
  • The more context a log/metrics event has, the greater its utility. Cribl helps users add rich context to events with data from other sources, like DNS, Threat Intel Lists, AWS, or Service Now at ingest time so that downstream analytics system can extract the most insights possible. Also, real-time alerting, to systems like Phantom, VictorOps, PagerDuty and event-driven platforms  now becomes possible and meaningful.

• Security & Data Privacy 
  • Cribl enables admins to obfuscate and encrypt sensitive information in their logs but retain uniqueness so analysis is still possible. Programmatic capabilities help with surgically targeting only valid strings for redaction/obfuscation. Encryption of data with role-based decryption is also available.

• Data Routing and Delivery 
  • Enterprises typically use multiple backends to store operational data. Some events belong in a real-time system, some others may need to be routed to a batch analytics store, and yet another portion may be sent straight for archiving. Cribl allows you to take any data collected from anywhere and reliably deliver it to where it will provide maximum value to your business; Splunk, NFS, Hadoop or a S3-compatible object store.

Pricing

Cribl is priced by daily ingestion volume, similar to Splunk, and we offer tiered pricing and discounts based on volume tiers. Cribl is free below 100GB a day, so anyone looking to get started should be able to build out a few use cases before ever needing to talk to us. All pricing is currently preliminary as we feel out the market and get an understanding of the value we’re delivering to our customers. We encourage you to grab Cribl, get some value, and then we’ll figure out together what that’s worth to your business.

Deployment Model

Cribl is software that deploys on your enterprise infrastructure, either on-prem or on Cloud. If you’re deploying it in a Splunk environment, there are two options; Cribl can be deployed on heavy forwarders or on indexers. The recommended choice will depend on your exact requirements and architecture, but for small environments, simply install and configure it on an indexer.

Processing Model: Routes, Pipelines and Functions

Cribl’s event processing model is very straightforward: As events come in, routes apply filter expressions and send matching results to the appropriate processing pipeline. Pipelines, are an ordered list of functions that work on the data serially. A function is code that executes on an event and it encapsulates the smallest amount of processing that can happen to that event. After events are processed, they exit the pipeline and get delivered to one of the supported destinations.

Functions:

Eval, Sampling, Lookup, Mask, Drop, Regex Extract, Regex Filter, JSON Unroll, Clone, Tee…and more coming

Destinations:

Splunk, Filesystem/NFS, S3 (or S3-compatible) …more coming.

If you are interested in Cribl, please head to cribl.io and download your copy to get started. If you’d like more details on installation or configuration, check out our documentation. If you need assistance, please join us in Slack #cribl, tweet at us @cribl_io, or contact us via hello@cribl.io. We’d love to help or hear your feedback!

Enjoy it!  — The Cribl Team

The fastest way to get started with Cribl Stream is to sign-up at Cribl.Cloud. You can process up to 1 TB of throughput per day at no cost. Sign-up and start using Stream within a few minutes.