Getting Started with Cribl Stream: Your First Hundred Days

Congratulations, you’ve worked hard to get Cribl Stream into your technology stack. Buying a new tool is a non-trivial task, so be sure to pat yourself on the back. Now the work starts: You have to deploy Stream and get full value to justify the cost. It’s critical to get started with the right plan to accelerate delivery and maximize the value of Stream.

I’m going to start by sharing some ideas about how to get started with Cribl Stream in your first hundred days. Future blog posts in this series will help evolve your deployment and take Stream to the next level.

Always remember the immortal words of Warren Buffet:

“Price is what you pay; value is what you get.”

Your job is to extract as much value as possible to justify the tool investment. With some thought and planning, you can extract far more value than the price you paid for the tool. We’re here to help you in any way we can.

Summary

• Initial deployment planning
• How to prepare your team
• How to keep your manager in the loop

Deployment Planning

Nothing happens until the infrastructure to support Stream has been deployed, but don’t make that hardware request until you know what you want to accomplish with your initial deployment. I highly recommend following the crawl, walk, run pattern so you can build value fast and keep growing your value delivery as your deployment gains momentum.

It’s critical to adopt a narrow scope for your initial deployment to allow a broad scope for your vision. Start with a chunk of data you know that has narrow complexity; this will allow you to get set up quickly, limit your variables, and learn valuable lessons before you move on to other data. Going too big can delay proving value and limit early feedback on any issues with your deployment. Go big on phase 2 after you learn lessons from phase 1.

I recommend getting started with syslog because, while it can be high volume, it’s fairly simple data. Your variables are limited, so you can make progress and show value faster.

Start by determining requirements:

1. What resilience is required to support ingesting this data?
2. Where does this data need to be delivered?
3. Can this data be transformed?
4. Does this data need to be enriched?
5. Who needs or has access to this data?
6. What is the data retention requirement?

Use the answers to these questions to formulate your deployment strategy. Resilience requirements are critical points to consider because they will drive your hardware requests. You don’t want to underestimate your needs and then lose data when something fails. Everything fails, so plan accordingly.

When you have answers to your questions, you can start requesting hardware. Start with the Cribl Sizing documentation, and don’t forget to request new load balancer pools to ensure you can scale your deployment horizontally and fail over when servers fail. Not if they fail, when they fail. Always plan for failure.

I’ll go into more detail in a future blog post. We’ll beat the subject of syslog to death since it’s something almost everyone has to manage and most struggle with it.

Preparing Your Team

The first step is, everyone goes to training. Thankfully, Cribl University makes this super easy. The user and admin training is also free, which is highly unusual and very welcome when you’re the manager and your team needs training. Have everyone on your team take both courses so you can start to build up a skillset to take full advantage of Stream. Scale your team so everyone can contribute. Most teams have uneven levels of expertise, which can create bottlenecks. For example, a team might have the syslog person or the regex person, or the person who really knows props and transforms.

Use this opportunity to get your team ready so everyone can enable syslog and everyone can build simple pipelines. Your backlog will appreciate this foresight as you get more work done with less effort, without waiting on your experts.

Also, start thinking about how you’ll manage your Stream code. Stream is a development platform, and you want to think carefully about how your team is going to write and manage code. I’ll go into more detail in an upcoming blog post, but below are some principles to consider as you think about bringing development principles into your team.

I highly recommend the following:

• Adopt Git or something Git-compatible as soon as possible. Don’t use local code-management options in Stream. Git enables a host of functionality and, more importantly, resilience if something bad happens.
• Regular code reviews are very important. You want everyone to understand what work is being done and know about any changes, because everyone is also on call and has to support Stream. I recommend twice a week, and make it mandatory. Use your afternoon standup or do code reviews twice for teams in other timezones. Get in the habit of not doing work in a silo.
• Adopt coding standards so everyone is writing code in a uniform manner. Be sure to require function-by-function comments. Link to your wiki for what will not fit in the comment box. Publish your code standards on your wiki and enforce them. Easy-to-read, well-documented code is easier to support and will save you when something goes wrong.
• Appoint someone on your team or yourself to be the product manager. This person will run the code reviews and look for issues with standards. For example: Are all functions documented? Does the flow make sense? iIs the code readable? I used to ask myself, “What would this code look like at 4 am on a bridge? Would I understand it?” Empower your product manager to say no, and require code changes before anything is deployed.

I’ll go into more detail in a future blog post.

Communicating Up

Bragging on your teams and yourself is a critical part of communicating up in any organization. You have to put a lot of thought into making sure everyone knows how awesome your team is and, by extension, how awesome you are. Your success does not matter if no one knows about it. Self-promotion is probably not your favorite way to spend time, but it’s as critical to being successful as the technology itself.

As a manager, you want to be ready to communicate status and share your vision for your tools on short notice. Build out a place in your wiki to share status and vision with your stakeholders and anyone else who can offer support now or in the future.

Start with a mission statement. Describe in as few words as possible how your work is benefiting the business. For example:

“The enterprise logging and analytics team seeks to unlock the value of all observability and security data for ‘Company X’.”

Focus on benefits to the business. Instead of tools or technology, focus on business value. How does what you’re doing add value, save money, or add new capabilities? Your message style matters because the business leaders may have no idea about the technology you use or even care, but they understand terms around business value. Build a reputation for delivering value, and you will find it easier to get funding for your next project and build your profile as a leader in the business.

Have a slide deck that introduces your team, what it does, and the value it delivers to the business. Also, know your elevator pitch so you can deliver your message on short notice if you need it in a meeting. Be ready for the moment when an executive says they have a challenge that needs a solution.

I’ll go into more detail in a future blog post.

What’s Next in This Series

Bookmark this page as we’ll keep the links up to date and provide you with the latest resources to help you on your journey with Cribl Stream. Future blog posts will include but not be limited to:

1. Observability and security data strategy
2. Deployment strategy
3. Getting started with Syslog
4. Conquering syslog
5. Using development concepts to manage your codebase
6. Adding log shippers to the mix
7. Scaling out to support your business units
8. Supporting operations and security use cases

Bottom Line

Deploying a new tool is a very exciting time. You want to take your new toy for a spin, but it’s important to put in the right planning so you’re not wasting engineering time. Thoughtful planning will save you a ton of time in the future. I look forward to sharing new posts in the coming weeks as we take the journey of deploying Cribl Stream together.

I’d love to hear your feedback on getting started with Cribl Stream. Feedback is a gift, and I want to know if something doesn’t make sense or if I’m not covering something. Connect with me on LinkedIn or join our community Slack, and let’s talk about your experience deploying Cribl Stream.

The First 100 Days with Cribl Stream Series

• Part 2: Start at the End to Progress Faster