Living the Stream

As the tech lead behind Data Collection, the leading feature of LogStream 2.2, I can say we faced many challenges developing a scalable, accurate solution to batch collection of data at rest.  Among those issues was a resource load balancing issue that cropped up late in our development of the feature.  We’ll discuss this issue […]

Scaling up with NodeJS This is the second post in a series aimed at describing how we designed Cribl LogStream to scale to process 10s of PBs of machine data per day. If you want to read why such scale is important check out part 1. In this post we’ll focus on scale-up, that is […]

One constant in log use cases is that you can’t plan for what you’re going to find at customers. Whether it’s multiple levels of encapsulation, like JSON-in-XML-in-Pipe-Separated (yes we’ve seen this), a need to radically transform the structure of events in a way we haven’t seen, or a need to reach out to an external […]