A Journey to Observability: Following Your Data From Generation to Analysis

I’m launching a new Observability Series called the Observability Professor, and it is designed to cover some common topics and terms in a vendor-agnostic way. That’s right, no marketing! So what’s special, what’s new, what’s it going to cover that everyone else in the industry missed?

Background: There are endless amounts of blogs, papers, and books on observability; what it is, and what it offers. On top of that, there are dozens of companies pitching their ‘perfect observability solution’ that will solve all your network reliability, performance, and security challenges with a simple P.O. That being said, I hope I can entice you to invest 30 minutes to listen to some educational content to help you thrive in the observability space.

How about Amara’s Law? “We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.”

I like this quote, and if you think about it you see similarities in Gartner’s Hype Cycle– it all starts with excitement and promises (the marketing), followed by reality (the disillusionment), and finally objectivity (realized value). That is what Observability is becoming to many people. But maybe it’s time to step back and view Observability from a different perspective. The truth: observability is not a technology, a product, or even a solution. It’s a methodology. Think along the lines of a multi-layered construct that leverages multiple technologies and tools to assist you in understanding your network or organization’s operation.

I like using analogies and a great one for Observability is network security; specifically the onion approach to IT and Security — different layers, different products. So using the security analogy, you’re in the market for a security solution, but you can’t just go out and buy ‘security’, you can only purchase security components, such as firewalls, VPNs, DLP, etc. It is then up to you to architect and build a solution, out of individual components, to address your unique security needs.

In a previous career in aerospace, all my work was in a SCIF (Sensitive Compartmented Information Facility). Then I worked with public Wi-Fi networks– significant differences in requirements, but both involved ‘security’. Well, the same goes for your approach to Observability. There are no out-of-the-box, one-size-fits-all solutions for Observability because what your organization requires for security or Observability is going to be very different from another organization’s requirements, and they will continue to evolve.

That is what this Observability Series is all about! It is designed to present a different view of what Observability provides, by not focusing on a 2 or 3-sentence definition, but defining and examining different observability components– some you may need, some you may not. This will be a 10-module series. In the first module, I will introduce the series and define the journey we will be taking. The second module will focus on arguably the most important component of Observability — the Business and Technical Requirements. This is where you determine what you need to know, what data you are looking for, what to look for, what to collect, and what tools you will need to analyze the data. Pretty much the foundation of why you need an observability solution at all.

Following the intro module we will have a series of on-demand modules that examine different steps and components along the journey. These modules will focus on the data types, where they are generated or where they ‘live’, the processing tools, Observability pipelines, search tools, security needs, application performance, the different IT teams and what they are interested in, and other topics.

Why Should You Join Us?

Data volumes are huge and growing, but budgets are not. The result is the percentage of data actually being analyzed will continue to drop due to licensing costs. There are only two options to address this issue: get a bigger budget or be smarter about how data is processed prior to ingesting into the system of analysis– this means a deeper understanding of observability offerings.

What we will discuss:

• Why separating the system of analysis from the system of retention is a good idea
• Why some data can go right to analysis, some direct to storage, & some left where it is
• Why multiple search options are superior to one size fits all
• Why no single vendor can provide the solution you need
• How storing raw data in low-cost data stores is better than expensive analysis systems
• Why query data in-place (data stores), and only route relevant data for analysis

Ready to join me on an educational journey without any vendor marketing? Register Now!

For more information, visit our Search product page.