Observability: You Can’t Buy It, You Must Build It!

In Part 1 of this series, we talked about the origins of observability and why you need it. In this blog (Part 2), we will cover exactly what observability is, what it isn’t, and how to get started.

Before we can dive into how to approach observability, let’s get one thing clear: You can’t buy a one-size-fits-all observability solution. That’s because observability isn’t just an evolution of monitoring, although almost 100% of articles you read compare and contrast the two. Observability is a concept, a goal, and direction you want to drive your organization to gain the most insight from the data you can collect.

For me, the best comparison to network observability is network security, you can’t buy it, but you can build it. Sure, you can buy a new firewall to provide increased security, but a firewall is just a security component, not a full-fledged security solution. You wouldn’t compare security to a firewall because it isn’t really comparing apples to apples. The same is true when comparing monitoring and observability — you can’t say one is better or an evolution of the other. Firewalls are a security component, just like monitoring is a component of observability.

So, What Exactly is Observability?

Observability is better described as a methodology you incorporate into your enterprise architecture to provide greater visibility to what is occurring.

We’re at the point now in the lifecycle of observability where it’s become more of a marketing buzzword than a set of fully understood principles. Don’t believe me? Go to any IT industry trade show and you will see about 80% of the booths have observability stamped on the side. That being said, they are not actually wrong. The hype is real. After all, venture capitalists have invested over $2 billion in the industry to date and they don’t tend to waste money on just hype, so what’s it all really about?

Building Observability: Defining Your Specific Needs Then Fill in the Gaps

Because observability is a concept and not a specific product, you must first define what observability means to you and your organization, this means defining your needs and then selecting the components required — and just like all the best toys, some assembly is required — but the good news is that you’ll likely have some of the pieces already in place. So, If you have a Splunk, Datadog, Syslog, any of the hundreds of other observability tools from vendors, or even if you’re just using SNMP traps, you’re already capturing data. They’re all part of your observability solution.

One way to build your observability solution is to look at it the same way you would a security solution. If you’re in the market for improved security for your network, you can’t just ‘go out and buy it.’ All you can do is purchase security components that you will need to architecture to meet your unique security needs. That’s really how you should approach observability too.

As with security, there are no out-of-the-box, one-size-fits-all solutions for security or observability because what your company requires is going to be very different from the rest. Just like a mom-and-pop sandwich shop will have much different security needs than a bank, each organization will have its own approach and requirements for observability.

Observability Will Continuously Evolve

Once you figure out exactly what you need and collect all the components, build or buy, and get a solid foundation going for your observability architecture, you’re still not done. Your observability needs will continue to evolve The devices, applications, threats, and capabilities of the systems you deploy are dynamic, so continuous integration and continuous deployment are required. It’s best to think of your observability solution along the lines of an OODA Loop: Observe, Orient, Decide, and Act. You must constantly be taking note of and adjusting to the changing environments.

This is how observability will evolve. You’ll complete certain things, then start over each week, month, or quarter because there might be something you’ll need to adjust to. It might be a great new application launched. One of your colleagues in SecOps might need to learn a bit more about a threat, or someone from business intelligence might want to examine some specific data you have. Observability constantly evolves and grows, so setting things up right from the start is in the best interest of your organization.

Building Observability: Avoid Vendor Lock-In and Share Data Seamlessly

There are a lot of great tools out there, but getting locked into one vendor can make it rain on your observability parade. Every organization already has some existing observability infrastructure that involves multiple applications – as such, Cribl doesn’t believe in a rip and replace approach to your existing tools. We believe in complementing what you already have and then offering you the choice to add additional capabilities. If you have Splunk, DataDog, AWS, Elastic, Kubernetes, or any of the other dozens of applications we integrate with, our vendor-agnostic products like Stream and Edge sit right in the middle (…or right at the edge!) of your data sources and destinations to make sharing data easy.

Want to learn more? Check out our on-demand webinar: Observability: Everything You’ve Heard is WRONG (Almost).