Cribl puts your IT and Security data at the center of your data management strategy and provides a one-stop shop for analyzing, collecting, processing, and routing it all at any scale. Try the Cribl suite of products and start building your data engine today!
Learn more ›Evolving demands placed on IT and Security teams are driving a new architecture for how observability data is captured, curated, and queried. This new architecture provides flexibility and control while managing the costs of increasing data volumes.
Read white paper ›Cribl Stream is a vendor-agnostic observability pipeline that gives you the flexibility to collect, reduce, enrich, normalize, and route data from any source to any destination within your existing data infrastructure.
Learn more ›Cribl Edge provides an intelligent, highly scalable edge-based data collection system for logs, metrics, and application data.
Learn more ›Cribl Search turns the traditional search process on its head, allowing users to search data in place without having to collect/store first.
Learn more ›Cribl Lake is a turnkey data lake solution that takes just minutes to get up and running — no data expertise needed. Leverage open formats, unified security with rich access controls, and centralize access to all IT and security data.
Learn more ›The Cribl.Cloud platform gets you up and running fast without the hassle of running infrastructure.
Learn more ›Cribl.Cloud Solution Brief
The fastest and easiest way to realize the value of an observability ecosystem.
Read Solution Brief ›AppScope gives operators the visibility they need into application behavior, metrics and events with no configuration and no agent required.
Learn more ›Explore Cribl’s Solutions by Use Cases:
Explore Cribl’s Solutions by Integrations:
Explore Cribl’s Solutions by Industry:
Watch On-Demand
3 ways to fast-track your data lake strategy without being a data expert
Watch On-Demand ›Try Your Own Cribl Sandbox
Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Get inspired by how our customers are innovating IT, security and observability. They inspire us daily!
Read Customer Stories ›Sally Beauty Holdings
Sally Beauty Swaps LogStash and Syslog-ng with Cribl.Cloud for a Resilient Security and Observability Pipeline
Read Case Study ›Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Transform data management with Cribl, the Data Engine for IT and Security
Learn More ›Cribl Corporate Overview
Cribl makes open observability a reality, giving you the freedom and flexibility to make choices instead of compromises.
Get the Guide ›Stay up to date on all things Cribl and observability.
Visit the Newsroom ›Cribl’s leadership team has built and launched category-defining products for some of the most innovative companies in the technology sector, and is supported by the world’s most elite investors.
Meet our Leaders ›Join the Cribl herd! The smartest, funniest, most passionate goats you’ll ever meet.
Learn More ›Whether you’re just getting started or scaling up, the Cribl for Startups program gives you the tools and resources your company needs to be successful at every stage.
Learn More ›Want to learn more about Cribl from our sales experts? Send us your contact information and we’ll be in touch.
Talk to an Expert ›Ed Bailey is a passionate engineering advocate with more than 20 years of experience in i... Read Morenstrumenting a wide variety of applications, operating systems and hardware for operations and security observability. He has spent his career working to empower users with the ability to understand their technical environment and make the right data backed decisions quickly. Read Less
The process of adding new data to operations and security analytics tools is familiar to admins. New data onboarding can be a tiresome process that takes up too much time and delays getting value from the new data.
The process typically begins with the admin engaging the data source owner, getting the wrong data sample, and then having to try again. Once the correct data sample is provided, then estimate the impact on computer/license resources, write the parser, test it, and finally deploy it into production. This process is frankly a pain. My goal is to discuss why an onboarding process is necessary and how to reduce friction and make it more tolerable – enjoyable, even.
Your team has now fully deployed Cribl Stream and is eager to begin solving problems. Instead of running to build code, now is the time to pause and put your processes in place to make your efforts sustainable.
Start with the log onboarding process. In a great blog post, Jon Rust wrote about how to build event breakers for new log sources in Cribl. For anyone who has had to build event breakers/parsers at the command line, his post is eye-opening for how easy it is to build event breakers using Cribl Stream’s UI. Even better, you can validate the event is breaking properly using the UI. For more details, see the embedded video in the blog. This is one of the most useful posts you will see.
I want to focus on why the onboarding process is so important and how to materially lower the friction of the normal process to something manageable so your engineers can spend more time on more business-critical tasks.
Cribl Stream’s default event breaker is attached to the data source, which can make it easy to miss if you’re not looking for it. Sometimes admins assume Cribl Stream is just passing events through without parsing and that can lead to problems. Although the default event breaker is very very flexible, it might not work correctly for custom events, especially application data, which is why it is critical to validate every log source with the default breaker and build a custom breaker if required. In addition, make sure you validate the timestamp and define the timezone as well. Nothing is as “fun” as finding a device’s events are exactly 9 and half hours off current time because the device is in IST and you are in EST and someone forgot to account for the timezone offset.
Use the onboarding process to document your data as well. Documenting your data matters as you try to understand what you are logging and what it means to your business. This is a big help when you need to quantify the value of your data. Another big benefit is having these docs for when the audit/records team appears from a puff of smoke when you are really busy and do not have time to answer questions.
Cribl’s Jordan Perks has a great format for documenting each data source.
Use this form to also update your license, storage, and compute forecasts so you can manage your capacity and budget accordingly. It is so important to stay on top of your capacity forecast so you don’t get caught short.
Finally, onboarding your data source by source is a great way to start thinking about how you can make the data better. This includes dropping, sampling, enriching, and format transforming. The best part is you can transform your data to make it better and smaller. Have your cake and eat it too. Here is a great video about the Cribl Windows pack which makes it easy to make Windows data 33% smaller without losing any fields or data. The ability to transform data from ugly to useful with minimal effort is part of what makes Cribl Stream unique.
I do not recommend making these changes at this point. You want to onboard the data and get it into your systems first and then come back to make changes. Small steps will get you to value faster and give you the necessary foundation to make the right decisions. Doing too much creates delay and slows down value. Schedule the next steps in your security and observability data
Here’s another good video with examples to start optimizing your data.
A consistent data onboarding process is key to long-term success.
I am going to discuss steps 4-6 in my next couple of 100 days with Cribl blog posts. I am particularly excited to talk about how to automate log onboarding while still retaining control of your data and preventing a free for all.
I’d love to hear your feedback on getting started with Cribl tools. Feedback is a gift, and I want to know if something doesn’t make sense or if I’m not covering something. Connect with me on LinkedIn or join our community Slack, and let’s talk about your experience deploying Cribl Stream.
Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.
We offer free training, certifications, and a generous free usage plan across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started. We also offer a hands-on Sandbox for those interested in how companies globally leverage our products for their data challenges.
Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.
Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari
Got one of those handy?