Wow, can you believe it’s been a year since we announced Cribl Search and redefined the search process for observability and security data? Even though the product is just wrapping up its first year, Cribl Search has achieved unrivaled growth in both user adoption and expansion of product capabilities. Why? the biggest reason for this success is that Cribl was founded on the basic principle of customer first, so we design and build products for our customers to solve real problems they are experiencing.
The perfect example of this approach was Cribl Search, the first product to introduce our search-in-place and federated search capabilities, along with an agnostic search engine that enables administrators to query almost any observability and security data from a single interface. Essentially, Cribl Search flipped how administrators search their datasets. Instead of the traditional way of collecting, routing, ingesting, and only then searching, administrators are now able to dispatch the queries to where the data is located. Cribl Search is engineered to let you search data-in-place, whether the data remains at the edge, in stream, in an observability lake, or still on the endpoint that generated it.
Ok, we all know that we are now able to generate and collect far more data than we can ever afford to analyze effectively; the result is that some enterprises report utilizing less than 2% of collected data. The other 98% typically gets routed directly to storage to review later. But, in reality, this data is deteriorating in value as it sits. Its ability to answer critical security, performance, and system state questions quickly fades. This led to the birth of Cribl Search, which continues to evolve to address this challenge.
Day 1 – Search provides out-of-the-box search-in-place capabilities, allowing administrators to query any data in any format at multiple locations via Federated Search capabilities. So, data stored in AWS S3 buckets, or even located on edge devices, is now within reach. This vastly increases the scope of analysis. No longer requiring the cost or complexity of first having to collect, route, ingest, index, and store the data. But that was just the beginning of the evolution of Search. And we had no intention of sitting on our laurels.
Before we even released Search, we already had the following year of innovation planned out and started releasing new features every month as they become available. Cribl Search, being a cloud-based SaaS application, had additions and enhancements automatically updated with no action required by users.
Winter Q1 Cribl Search Additions
To be honest, our first-generation users loved the concept of Cribl’s search-in-place but were looking for an additional number of original datasets they could access. The ability to search data in AWS S3 and on other host platforms was a great start, but they were quick to request access to additional data stores and resources. We listened.
The following quarter, we delivered a 500% increase in the number of dataset providers that could be queried. To go along with AWS S3, we added the ability to search Azure Blob, Google Cloud Storage, and even the brand-new Amazon Security Lake. Plus, data doesn’t just live in lakes, and with all the SaaS users have deployed, we included the ability to query live APIs now, too. This included access to endpoints from many common services such as AWS, Okta, Zoom, GCP, Google Workspace, Microsoft Graph, and more.
While administrators loved access to more data sources, manually running queries was too time restrictive and consuming. This led to adding automated searches, allowing query results to be waiting when you arrive in the morning. Plus, the ability to aggregate the data over time.
Since most of our Search customers were existing Cribl Stream users, they wanted (demanded) greater interoperability between the two. We added the ability to send Search results directly to Stream for additional shaping and forwarding. To make this process easier, we allowed users to append one word to any query (‘send’), and the results auto-magically appear in Stream.
More “traditional” updates that focused on enhancing and simplifying the user experience. This included improved parsers to services, support for additional data types and formats, and adding additional operators and functions to focus the query and shape the results.
Spring Q2 Cribl Search Additions
Q1 was full of new features, but customers weren’t done asking, and the product team wasn’t done delivering. No rest for the weary (our engineering teams).
You would think a 500% increase in available data set providers was enough. But, noooooo we needed more. Over the next few months, we added additional access to Azure, Tailscale, and even external data (this is a really cool one; check it out).
Limiting who can do what? Who would’ve thought security would be a concern? Actually, we already did. Just as the requests were coming in, we rolled out Authorization Services with new members and permission levels to provide finer-grained access control to the datasets, resources, and results. Allowing more users but guardrails to accessible resources.
Hey, what about visualization? Another one we already had teed up, and one day, a new dashboard tab appeared on the screen. Now, users can easily customize visual displays of search data with the ability to create, manage, and customize results. Don’t forget about various widget types and visualizations that give the ability to tailor dashboards to best fit specific requirements.
Hey Cribl! Scheduled searches are great, but what if an automated search detects an abnormality in the middle of the night or over the weekend? Do I really need to wait until the next time I log in to see it? Of course not! Notifications optimize the schedule search functionality, adding the ability to alert if a condition by defining triggers and sending notifications via PagerDuty, Webhook, and others.
The Search tool is great, but folks didn’t want to have to train other system administrators to use it. Boom, done ✅. The ability to access Cribl Search directly via its API interface – simplifying the integration with 3rd party products such as Grafana.
A common theme is that the more features we include, the more folks ask for more (for real – we love it because it helps us make our products better!). So, once again, we added more operators, functions, and data enrichment services with access to lookup tables.
Summer Q3 Cribl Search Additions
Q2 was a busy time, with improvements across all areas of the product. We continued to add new features while at the same time enhancing existing capabilities.
One of the asks was the ability for an administrator to quickly see what is going on at any time. With access to more datasets and the ability to regulate individuals’ access rights, at any given time, there could be multiple queries running and/or staged; how is the administrator to know what’s happening? We added new Show and Cancel commands, allowing administrators to identify objects in a dataset, see what queries are running, and even stop one or more running or queued queries.
I’m running out of breath at this point, the features just kept coming! More features were requested, so more adjustments were made, including more operators (like print), Lookup, event stats, Dedup, and export. Added Slack and Amazon Simple Notification Services as notification options and finally added improvements to dashboards, customizable time and history settings, virtual tables, and more.
Fall Q4 Cribl Search Additions
Without a doubt, 2023 has turned out to be an amazing first year for Cribl Search and its users. BUT we are not done yet!
Just this month, more operators, functions, and commands to improve shaping the query and the results were added. Plus, the increased ability for administrators to control users and not only access but control resource utilization, too. Finally, we have done some backend tinkering so Search queries are even faster, resulting in less waiting and less cost too. Read all about it here.
So What’s Coming in 2024?
Sorry, legal says I can’t let the cat out of the bag just yet. But you can expect access to even more data resources with a greatly expanded ecosystem. Also, think higher performance, expanded integrations, and a simpler UI. That’s right – more powerful and user-friendly, too!
If you’re ready to join the celebration, click here to learn more about Cribl.Cloud and how to gain instant access to Cribl Search!
