The Cribl blog covers Observability, Big Data Analytics, Data Streams Processing... and anything else we feel like writing about!
An ex-colleague at Splunk asked me in a LinkedIn post if Cribl LogStream does anything else besides log reduction. This blog is for him. LogStream optimizes data so that it’s consumable again. In this blog, I’ll focus on using LogStream to improve Splunk performance for search while lowering CPU usage. If you’re in the David […]
Since the inception of Cribl LogStream, we’ve had a freely accessible product. LogStream has always been available as a download or a Docker container. Since we launched version 2.0, I’ve often said LogStream is the easiest distributed system to setup I’ve ever encountered. Set up a leader node , set up one or more worker nodes, point them at the leader, and you’re good to go. If you’re running Kubernetes, we provide easy-to-use Helm charts for getting up and running quickly.
Recently, I had the opportunity to work with a customer who was looking to reduce their Splunk license cost. They were looking to expand their use of Splunk, but were constrained by the growth of their data volumes, and couldn’t spend more on top of their 500 GB license currently in use.
You may want to keep more data in storage for longer periods of time, but the costs associated force difficult decisions. How much data should we collect? How long should we keep it in storage? Where should we store it? To understand how a storage unit for observability data might work, let's explore an analogy about movie night.
In part one of this blog post, I covered the concept, basic design, and results of using Redis to enrich VPC Flow Logs with security classification data from the GreyNoise API. In this post, I’m covering the details of how to do it. These steps will get you going if you want to try it, but keep in mind you’ll need your own GreyNoise API to run it.
Data collection from Amazon S3, first introduced in Cribl LogStream 2.0, has been an overnight success with most of our AWS customers. In 2.4.4 we’ve added a similar capability to read data at scale from Azure Blob Storage, where a lot of other customers store massive amounts of observability data; logs, metrics, events, etc. In this post, we’ll take a look at how it works, and how to configure it.
Postgres, like many database applications, has a robust dynamic trace capability. Combined with a highly configurable log facility, it’s quite possible to track database activity. But as with most attempts at observability, it isn’t quite that simple. AppScope has the ability to track all SQL activity associated with a Postgres service.
Previous experience with Protobuf was just painful, to be honest. How complicated is this? Worth doing? All of which caused me to think about how to analyze gRPC. Since AppScope extracts payloads from network activity, could we see gRPC and Protobuf details?
To deal with this tool sprawl, many enterprises chase after a “single pane of glass” strategy, where a single tool offers all the capabilities various teams need. According to 451 Research, 83% of enterprise companies prefer buying as many monitoring tools as possible from a single vendor. While this sounds like a great strategy, there are several reasons why a single-vendor strategy doesn’t work.
We are increasingly asked if a Cribl LogStream instance can be used to send to another LogStream instance. The answer is a definite yes! While there are various reasons for wanting to send data from one LogStream instance to another, this blog explores one example in depth and describes various protocols and other considerations.