Search is where observability platforms either create momentum or create friction.
Teams collect more data than ever, but when an engineer needs to investigate an incident, speed matters more than storage. If a search for a single error token forces a broad scan across massive log volumes, time-to-answer stretches, infrastructure costs rise, and the platform starts to feel heavier than it should.
That is why full-text search matters.
In Cribl Search’s Lakehouse Engine, full-text search uses an inverted index on _raw, the full event message, so searches for alphanumeric tokens can be dramatically faster and users can search the whole event without needing to target a specific field. In our latest 4.19 release (July 2026), that capability became the default experience for Lakehouse engines, including backfill support for existing engines.
The problem with field-dependent search
A lot of search experiences work well only when the user already knows exactly where to look. That sounds fine in theory, but it breaks down in real environments.
During incident response, analysts often start with fragments:
an error code
a hostname fragment
a process name
a user identifier
a suspicious string copied from a ticket or alert
Those clues frequently appear first in the raw event message, not in a perfectly extracted or accelerated field. Requiring users to pre-model every possible search path creates operational drag. It also slows down the first phase of investigation, which is usually the most important one.
Full-text search changes that model. Instead of forcing the user to guess the right field first, it lets the platform optimize for the way people actually hunt through data: starting with a token and following the evidence.
How Cribl approaches full-text search
The underlying implementation uses a text index on _raw with tokenization based on non-alphanumeric boundaries and lowercasing during preprocessing. In practical terms, that means the engine can quickly locate word-like tokens inside the event message rather than scanning every row end to end.
That architectural choice matters because it aligns with how users search logs in the real world. Many investigations begin with “needle in the haystack” behavior: find the one string, code, or name that reveals where to go next. The internal design notes explicitly frame the benefit around those scenarios and show how the index helps target the specific areas in storage where the desired data is likely to live, drastically increasing search performance. .
The result is a search experience that feels more natural for exploratory investigation and more efficient for the platform underneath it.
What the performance data shows
The most compelling part of full-text search is not the architecture diagram. It is the outcome.
In internal testing on a 1 billion row dataset, the indexed approach delivered acceleration ranging from about 150x to 350x for token-based “needle in the haystack” searches, depending on selectivity. In fact, the more rare the search term, the greater the acceleration provided!
That kind of improvement changes the user experience in a meaningful way:
the first search comes back faster
exploratory iteration becomes easier
users can search the full event message without first building a perfect field strategy
compute is spent on narrowing the answer, not scanning everything
This is especially valuable for technical buyers evaluating total platform efficiency. Faster search is not only a usability feature. It is also a cost and architecture story. When the engine can skip large amounts of irrelevant data, it reduces the amount of work required to answer the question.
Full-text search does not replace good data modeling
One of the more interesting findings from the internal work is that full-text search and indexed fields complement each other.
The testing showed that combining full-text search on _raw with indexed fields helps address parsed-event scan performance concerns. Even in scenarios where the query still needs structured predicates, the text index can narrow the candidate set before those additional filters are applied.
That is an important message for buyers and architects.
Full-text search is not a replacement for thoughtful schema design, enrichment, or field extraction. It is a force multiplier. It improves the first step of discovery while still working alongside the structured access patterns that mature teams rely on.
A practical way to think about it is this:
use full-text search to find the signal fast
use fields and structured predicates to refine, correlate, and operationalize
That combination supports both speed and rigor.
What users should understand about search behavior
There is also an important nuance in how the index behaves.
The full-text index matches on token boundaries, which means it does not natively support arbitrary in-string matching the same way a raw substring search would. Currently, the full text search index is chosen by the search optimizer when a simple string (or several) is included in the left-most implied cribl operator, or when the where operator is used with the has string operator. No asterisks or wildcards may be included in the strings.
Some examples:
dataset=”flowlogs” 123456 | ... // successfully uses the full-text search feature
dataset=”flowlogs” | where _raw has 123456 | ... // successfully uses the full-text search feature
dataset = “access_common” field1=”nogood” | ... // does not use the full-text search feature, because a field name was explicitly called out in the predicate
dataset = “access_common” *foobar* | ... // does not use the full-text search feature, because wildcards are not allowed as part of the predicate
dataset = “access_common” | where _raw contains ”foo” | ... // does not use the full-text search feature because the contains string operator allows substring matches
All of the searches that do not use the FTS index will still succeed, however they do not benefit from the acceleration provided by the full-text search feature.
Follow-on engineering work may include improving support for operators like contains, startswith, endswith, and wildcard-style matching through query rewrites that still take advantage of token-aware skipping.
This is a useful point for technical evaluators because it shows maturity in two directions at once:
the platform is optimizing for the search patterns that matter most
the team is actively improving operator coverage where user expectations extend beyond simple token matches
In other words, full-text search is powerful, but it is most effective when users understand the difference between token search and arbitrary substring semantics.
Why this matters to technical buyers
For teams evaluating modern data platforms, full-text search is not just another feature checkbox. It is a proxy for platform philosophy.
It tells you whether the system is designed around the real behavior of operators and investigators, whether performance has been engineered into the search path, and whether the vendor understands that fast answers create trust.
Cribl’s approach is compelling because it makes the default search experience more aligned with the way people actually work:
start with a clue
search the full message
get results fast
refine from there
That is a better workflow for incident response, troubleshooting, ad hoc analysis, and general operational exploration. And when that workflow becomes the default rather than a special tuning exercise, the platform becomes easier to adopt across teams.
Final thought
Full-text search matters because modern log analytics should not force users to choose between flexibility and performance.
When the platform can search the full event message quickly, users spend less time wrestling with the system and more time solving the problem in front of them. Cribl’s Lakehouse full-text search work shows what that looks like in practice: faster token-based searches, broader access to the full event message, and a default experience designed for how investigations actually begin.
