The best part of my job is talking to you, our prospects, and customers, about your logging and data practices. I love listening to what you are doing and hope to accomplish, so I can get a sense of the end state. My goal is to brainstorm solutions that provide overall value across the enterprise, and not just aim for a narrow tactical win with limited impact.

In late September, I hung out at a local DevOps conference in Brooklyn with the NYC Cribl sales team. We got to talk to DevOps and security teams about their data challenges. I don’t run into as many DevOps teams as I would like, so this was a real treat. I got to engage them and gain a deeper understanding of their needs. Three major challenges recurred in our conversations:

1. DevOps teams are struggling to provide quality logging to help troubleshoot problems and secure applications with the right data. Logging development work is time-consuming and a lower priority than delivering new features.
2. Kubernetes exploded into the scene and is generating massive amounts of data, weighing down free tools with the deluge. In addition to data latency and storage costs, the inability to monitor across Kubernetes clusters is a growing concern.
3. Another hot topic was how to handle privacy regulations. Large companies operating in multiple countries are subject to various privacy regulations. How should the DevOps team collect, get value from the data, and stay compliant? These are big questions to unpack for everyone in the enterprise.

Struggling to Implement Best Practices

When some of the DevOps teams described their current practices, I could see why they were struggling. A common pattern was to map one Kubernetes cluster to a DevOps monitoring tool, like open-source Grafana, that was owned by one DevOps team. This pattern worked fine initially, but as Kubernetes sprawled and more teams adopted it, the breakdowns became inevitable. There were too many instances of Grafana that were siloed, and too many Kubernetes clusters. Hard to manage, and even harder to get a handle on monitoring how applications interact across different clusters. In this world of microservices, you have to be able to see the whole of the solution and not just the parts.

The Benefits of AppScope

Demoing Cribl AppScope was a revelation for the developers. The idea that they actually could instrument their applications with dramatically less effort, and generate clean detailed telemetry to solve a variety of use cases seemed impossible to them. I had fun showing them how to use AppScope from the command line to interrogate a Go binary that had no logging. How do you troubleshoot code issues with no logs? With AppScope, we were able to view everything the binary was doing. Developers got to understand why AppScope would be a great tool in their DevOps/SRE toolkit.

With Cribl Stream, you can collect and consolidate data from Kubernetes clusters to fewer observability tools to make it easier to manage storage, and correlate monitoring across clusters. Stream also can ingest data faster and reduces issues with ingestion latency. Even better if you use Cribl Edge to consume data directly from the Kubernetes clusters and reduce latency to zero. Edge loads as a DaemonSet so no more sidecars and less complexity. Everyone needs similar ways to get things done.

Finally, filtering/masking/redacting sensitive data is much easier with Cribl Stream. It gives options for staying compliant and out of the news. We demoed how developers can take complete control of making data safe for use, and only apply select filters to the right data. They liked the idea of keeping a complete copy of the data in compliant storage but then depersonalizing it to share for observability and analytics use cases and unlock the full value of the data. Max Schrems is busy making sure GDPR requirements will keep evolving and become more restrictive, and DevOps teams need options for handling current and future regulations.

At the end of the conference, we shared a fun happy hour at Harriets by the Brooklyn Bridge. It has an amazing view with great food and drinks. I was glad to spend time with so many interesting people with great stories.

Bottom Line

It was great to spend time with DevOps teams in Brooklyn. We met engineers and architects who know their challenges and are ready to look for new options. I look forward to working with the teams to help solve real-world issues and free up time to help innovate their security and observability practices.

Try Cribl’s free, hosted Stream Sandbox. I’d love to hear your feedback; after you run through the sandbox, connect with me on LinkedIn, or join our community Slack and let’s talk about your experience!