The team at Cribl is excited to announce the August 2021 updates to Cribl Stream. We announced it in our community a few hours ago, and we just couldn’t wait to make all of the details publicly available. Let’s dive into what’s new in our latest release.

Notifications Take the Guesswork Out of Data Reliability

An important part of a mission-critical observability pipeline is being able to guarantee the flow of data to multiple destinations. But you don’t always have control of those destinations. When something goes wrong downstream, it can be hours before the team responsible for the destination knows there’s a problem.

The Notification features in Stream enable users to identify a destination that is no longer receiving data, and to quickly notify the right team that the destination needs to be investigated.

You can send Stream notifications to your on-call system, ticketing service, or ChatOps channels, enabling data teams to respond quickly and collaboratively. While your teams investigate data destination outages, they can simultaneously prevent data loss and poor user experience for those providing essential reports, dashboards, and analytics services.

The Cribl Pack Dispensary™ Is the Grooviest New Content Library in Town

What good is a way to build and share Stream Pipelines, Samples, and Knowledge Objects – that is, Cribl Packs – without a place to share them?

Welcome to the grand opening of the Cribl Pack Dispensary! A community of data experts building and sharing Cribl Packs, to make working with streaming data easier, faster, and more valuable for everyone.

The Cribl Pack Dispensary integrates with the new Pack management tools that Stream added in 3.1.0. Install directly from any of the dozen (plus!) Packs are already available in the Dispensary.

Not able to connect to GitHub directly from your Stream environment? We got you. All of the Packs in the Dispensary have a .crbl file, accessible under Releases, that you can download directly and install in air-gapped environments.

For those looking to pull from private repos, branches, tags, and/or their own internal git systems: You can do all of those things with Stream 3.1.0 – giving you secure and flexible control of Pack development and deployment in your Stream environment.

Thinking about developing a Pack? Then join us at #packs in our Slack Community workspace.

Secrets Manager Available for Every Source and Destination

We’ve extended our Secrets Manager to work with every source, collector, and destination, making working with auth secrets, tokens, and username/password combos simple and secure. The Secrets Manager provides a central place to create, store (encrypted of course), and recall secrets for use in authenticating with all sources and destinations.

We’ve even thrown in the ability to call the Secrets Manager as part of certain Stream functions, using the new C.Secret() internal method. This makes it safe to use auth tokens and/or key/value pairs outside of sources, collectors, and destinations, without worrying about exposing those values in plaintext somewhere in your config.

Google Security Operations, Grafana Cloud, and Prometheus Integrations

What would we be without our rich ecosystem of upstream and downstream data partners? We’d be an inert pipeline floating pointlessly in the infrastructure void. Luckily, we have many integrations, so we annoy the void with direction and focus. In this release, we’re adding 6 new integrations (bringing our total sources, collectors and integrations count to well above 70), including:

• Grafana Cloud (destination for logs & metrics)
• Grafana Source (source for logs & metrics)
• Loki Logs (source & destination)
• Prometheus Remote Write (source)
• Google Security Operations (destination)

Each of these integrations provides an easy way to collect data, route it, and expand your observability reach – without sacrificing choice.

Stream customers want to use their existing streams of data to try new security tools, or to enable other teams with valuable observability data by sending directly to the products and services those teams use daily. Our ongoing addition of new native integrations serves both goals. Some examples:

The Google Security Operations destination makes leveraging your existing security events with Chronicle something that can be achieved in a matter of minutes. This provides threat hunters with the sheer volume of data they need to do their job – without having to re-instrument their entire infrastructure.

The Grafana sources and destinations extend Stream to provide a universal point for collecting, processing, and routing all your observability data. And, to make sure we covered all our bases, you can collect data from any Loki- or Prometheus-compatible source, and send it to any Loki-compatible destination. The same for the Prometheus metrics that are often used to power Grafana dashboards.

Other Items in the August 2021 Release

We’ve also added several updates to Stream, beyond what we’ve talked about today, including:

• The Code Function, for writing full JavaScript functions as part of a pipeline.
• Configurable behavior for what to do when Persistent Queues fill up. And multiple updates to the UI, to account for all the new stuff in Stream.

To get started with Stream 3.1 in your cloud, or ours, sign up for Stream Cloud. There, you can use Stream for up to 1TB of throughput per day, free on accounts created through Jan. 1, 2022. Or, download Stream and run it in your own environment. You can license more throughput as you need it.

Don’t forget to stop by the Cribl Pack Dispensary for a pick-me-up, and join us on Slack if you have any questions. Happy streaming!