At Cribl, we rely on our own products to tackle tough problems. I’m part of Cribl’s Customer Centric Engineering (CCE) team, a specialized group within Technical Support that bridges the gap between frontline troubleshooting and deep engineering analysis.
Think of us as Tier 4 support: we jump in when Technical Support Engineers (TSEs) run into complex issues that go beyond standard support—digging into bugs, validating feature requests, and analyzing log data from the inside out. Since joining the CCE team this past summer after already being at Cribl for five years, I’ve spent a lot of time digging into tricky support cases using Cribl Search to analyze log data and validate findings.
Before Cribl Notebooks, most of my work lived in Confluence pages — I’d paste in every query I ran, capture notes, and document each step then feed highlights into the Jira ticket to keep Jira comments as clean as possible. So Confluence served as the scratch pad. It worked, but it got messy fast, especially for investigations that involved lots of trial and error.
Now with Notebooks, I can see everything in one place — the data, the queries, the results, and the thought process behind them. I can pick up where another engineer left off or build on a Notebook that already exists, all while keeping the investigation transparent and easy to follow. It’s completely changed the way the CCE team and the larger Support team documents and collaborates on support investigations.
Communication and Collaboration Challenges
A lot of the issues I handle come down to communication and coordination — especially when data can live in multiple places. In the past, a TSE might drop queries and results into a Jira ticket to open the line of communication with the CCE team, and I’d have to pull those out and re-run them in Cribl Search to dig deeper. Now that TSEs can use Cribl Search directly within a Notebook, everything’s in one place. If the investigation is still fresh, I can even see their previous search results and build on top of them.
Worst case, I just refresh the data and keep going. It’s made our workflows much faster and our communication cleaner because the Notebook becomes the single source of truth for each investigation. We’re still building the habit across the team, but our goal is simple — instead of sharing queries or screenshots in Jira, we just point to a Notebook and keep the whole story there.
How I’ve Used Notebooks
When I’m digging into a performance or stability issue, I start with raw stat metrics like CPU usage, memory utilization, and per-process health indicators from relevant worker nodes. These give me a quick snapshot of system performance and whether anything’s over- or underutilized. Those data series are where I usually start before diving deeper into specific performance questions.
In one recent case, I used Notebooks to figure out why network connections were seemingly randomly closing between client devices and Stream worker nodes. By analyzing the relevant logs step by step, I was able to pinpoint the behavior — even though it wasn’t a performance issue. For other cases, I might use Notebooks to track metrics and system health, but the workflow is the same: run, document, analyze, repeat.
Cribl Notebooks has also made collaboration across our support team much easier. All queries, annotations, and observations are saved in a single Notebook, so anyone joining an investigation can quickly get up to speed without starting from scratch. It’s also a great tool for helping newer TSEs learn investigation best practices, and over time, we plan to build a library of case studies directly from Notebooks to share knowledge, speed up onboarding, and highlight interesting investigations.
Best Practices
Establishing Consistency
We’ve built some structure around how we use Notebooks to keep investigations consistent across the team. One of my support colleagues created an internal template that outlines best practices — how to name a Notebook, what kinds of queries to include, and how to annotate findings for TSEs to leverage. That consistency makes it easier for anyone to pick up an investigation and immediately understand what’s been done so far.
Building a story
When I start a new Notebook, I usually begin by dropping in the initial queries to pull the data I need. Once I have results, I go back and add annotations explaining what each query was meant to find. After reviewing the output, I’ll add another annotation with my analysis — what the data tells me and what I plan to check next. A single Notebook might include four or five queries by the time I’m done, each building on the last. It ends up reading like a story of how the issue was investigated from start to finish.
Keeping investigations visible
What makes Cribl Notebooks especially useful is how easy it is to collaborate — Notebooks are sharable by default, so anyone on the team can open one, see who last modified it, and pick up right where I left off. It’s become common for one engineer to create or update a Notebook and another to review or continue the analysis later, keeping the whole investigation visible and connected.
Tips and Looking Ahead
For anyone just starting with Cribl Notebooks, my advice is to really explore the Search interface and all the customization options. Many features that make results easier to read — like chart settings or display options — can be easy to miss because there is so much available within the rich interface, and taking the time to adjust them can make a big difference in how quickly you interpret data. Notebooks are a great way to highlight these adjustments, making search results and visualizations more accessible for yourself and your team.
Looking ahead, I’m excited to see how Notebooks can evolve. One feature I’d love is the ability to annotate charts directly — like a whiteboard where I could circle peaks, add arrows, or point to specific timeframes.
Cribl Notebooks has become an indispensable part of my workflow. By combining the querying power of Cribl Search with the flexibility of a collaborative, documented workspace, the team can now visualize data, track their investigative process, and build on each other’s work—all without jumping between tools or losing context in long Confluence pages.
**
Cribl Notebooks is now generally available! Try it today in Cribl Search. The only cost is for the searches you run inside them.
