These days, administrators typically have to deploy multiple tools to search through all of their datasets – then they get to spend the little free time they have left over dreaming of a world where they could search multiple distributed datasets simultaneously, similar to existing web search tools. They might have one tool for Splunk, another for Elastic, and some may even still be using grep or some other cumbersome function to search non-correlated data. The result is multiple tools, actions, and in some larger organizations, multiple employees to work with all of the different systems, time and money spent.

A Simple Observability Data Search Tool Can’t be Rocket Science

Most public search tools used for Internet searches can already retrieve information from a variety of sources via search applications built on top of one or more search engines. Users can make a single query and that request is then distributed to search engines, databases, or any other query engines that want to join the party. If you think about Google or any other tool you use, it already goes out and looks for information in a bunch of different places, displaying the combined results on a single screen. So why isn’t it the same type of tool available in observability?

Cribl Brings Federated Search to the Observability Space

The observability search tools of the past were so successful they ended up suffering from inertia, with only cosmetic changes to their processes. Cribl decided it was time for a change, so we took inspiration from Google’s search function and found a way to apply it to observability data. The result — Cribl Search, a shiny new, technology-advanced federated search tool ready for you and your system administrators to change the way you perform searches forever.

Cribl Search can federate the query to edge nodes, to S3, to any of your data, wherever location. Leaving you to sit back, relax and watch the traffic as it goes through. Most observability solutions don’t have this capability. If you want to search Splunk, you can go to their UI to search only what they’ve already captured. Elastic works in a similar way — they’re both great tools, but their search is limited to the data they’ve already ingested. What about all the other data spread across the enterprise, in your data lake? That’s where Cribl shines.

Query Multiple Datasets From a Single UI With Cribl Search

A dataset is just a bounded collection of data, a host, or multiple hosts, a S3 bucket or multiple buckets, you get the idea. The ability to query multiple datasets from a single UI is especially important when it comes to things that weren’t designed to be searchable like hosts, databases, or S3 buckets. And its capabilities go far beyond only being able to search data that’s already been collected — Cribl Search provides users access to literally all the data, wherever it’s located. It enables you to search the endpoint itself, giving visibility not only into the logs and metrics, but all files, including configuration files and system state information.

This includes everything those endpoints use to run applications or what they import to run their operations — and this is key because oftentimes it is not cost-effective to collect data from 100s or 1000s of hosts, to be routed back to and ingested into systems of analysis to see if there is any value in the data. Imagine being able to query the data, still on the edge devices and only if value is discovered, collect and analyze it. The ‘collect before search’ is so Gen-Z , if it was a song it would be relegated to the classics channel. But we have you covered there too, we can just as easily search collected data if it happens to be in a data lake or an index.

How Do We Access Data With Cribl Search?

Ok, a little deeper dive here, as mentioned we access data via datasets, a defined collection of data. Essentially, the dataset defines not just what’s to be queried, but also where to search, and how the information will be accessed, including any API keys or passwords you need to access it. With Search, you can also set access control rules to limit who can search your data.

Cribl Search ships out of the box with a bunch of predefined common datasets. You’ll be able to search within leader logs, worker logs, edge nodes, fleets of Edge nodes, and even S3 buckets, but you can also create your own too. There’s a helpful wizard available to guide you to simplify getting started, giving administrators the ability to define their own datasets and get up and running in no time.

What If You’re Not Sure Where Your Data Is?

If you know what you’re looking for, and you know where it is, searching is pretty easy — you just capture it all back and take a look. However, what if you’re not sure where your data is, or more commonly, what if the specific thing you’re looking for could be distributed across your enterprise, hosts, or data lakes – what’s the best way to search for that? One option is to hunt and peck, hoping you find something, but you’d probably have better luck going back 20 years and challenging Elon Musk to a rocket-building competition.

Instead, wouldn’t it be better if you could just select a dataset or multiple datasets? Maybe you want to look at specific hosts, workers, or AWS buckets. Or maybe your spidey sense is tipping you off that something crazy happened and you want to look for any instance of FUBAR within your data. Well, when you launch Search the focus can be as narrow or broad as you desire, as it activates query ‘engines’ where the data is. On the edge nodes or in AWS, these handle all the heavy lifting, searching through all the information and as far back as you need it. Then results are combined and displayed on that same UI, you can see the tremendous amount of data coming from different devices, all nicely correlated and with timestamps.

And if it’s just too much, you can simply identify information returned from the query to relaunch it again, performing as many iterations on the results as needed to get to the level required to meet your requirements.

This is a huge shift in the world of searching observability data, With the addition of Search, Cribl’s observability suite provides unprecedented capability. You can actually discover, collect, shape, route, and now search, all from a single UI — not a single pane of glass that has six or seven UIs behind it, but a single UI that shows you everything that’s going on. Your system administrators will thank you. Learn more about Cribl Search in our on-demand webinar.

Cribl Search Blog Series

• Searching Observability Data Just Became Point & Shoot
• The Most Powerful Tool for Querying Data at Its Source
• Search Observability Data In-Place: Store Where You Want, Query When You Want