Cribl puts your IT and Security data at the center of your data management strategy and provides a one-stop shop for analyzing, collecting, processing, and routing it all at any scale. Try the Cribl suite of products and start building your data engine today!
Learn more ›Evolving demands placed on IT and Security teams are driving a new architecture for how observability data is captured, curated, and queried. This new architecture provides flexibility and control while managing the costs of increasing data volumes.
Read white paper ›Cribl Stream is a vendor-agnostic observability pipeline that gives you the flexibility to collect, reduce, enrich, normalize, and route data from any source to any destination within your existing data infrastructure.
Learn more ›Cribl Edge provides an intelligent, highly scalable edge-based data collection system for logs, metrics, and application data.
Learn more ›Cribl Search turns the traditional search process on its head, allowing users to search data in place without having to collect/store first.
Learn more ›Cribl Lake is a turnkey data lake solution that takes just minutes to get up and running — no data expertise needed. Leverage open formats, unified security with rich access controls, and central access to all IT and security data.
Learn more ›The Cribl.Cloud platform gets you up and running fast without the hassle of running infrastructure.
Learn more ›Cribl.Cloud Solution Brief
The fastest and easiest way to realize the value of an observability ecosystem.
Read Solution Brief ›Cribl Copilot gets your deployments up and running in minutes, not weeks or months.
Learn more ›AppScope gives operators the visibility they need into application behavior, metrics and events with no configuration and no agent required.
Learn more ›Explore Cribl’s Solutions by Use Cases:
Explore Cribl’s Solutions by Integrations:
Explore Cribl’s Solutions by Industry:
Try Your Own Cribl Sandbox
Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Get inspired by how our customers are innovating IT, security and observability. They inspire us daily!
Read Customer Stories ›Sally Beauty Holdings
Sally Beauty Swaps LogStash and Syslog-ng with Cribl.Cloud for a Resilient Security and Observability Pipeline
Read Case Study ›Experience a full version of Cribl Stream and Cribl Edge in the cloud.
Launch Now ›Transform data management with Cribl, the Data Engine for IT and Security
Learn More ›Cribl Corporate Overview
Cribl makes open observability a reality, giving you the freedom and flexibility to make choices instead of compromises.
Get the Guide ›Stay up to date on all things Cribl and observability.
Visit the Newsroom ›Cribl’s leadership team has built and launched category-defining products for some of the most innovative companies in the technology sector, and is supported by the world’s most elite investors.
Meet our Leaders ›Join the Cribl herd! The smartest, funniest, most passionate goats you’ll ever meet.
Learn More ›Whether you’re just getting started or scaling up, the Cribl for Startups program gives you the tools and resources your company needs to be successful at every stage.
Learn More ›Want to learn more about Cribl from our sales experts? Send us your contact information and we’ll be in touch.
Talk to an Expert ›Are you looking to correct misconfigured timestamps from logs in your Security Incident and Event Management tool? Let’s say you’ve just received a critical security alert from the SIEM tool regarding a brute-force attack that occurred on July 19, 2021, at 11:08AM. The cryptic event looks like this:
The Incident Response team was dispatched to investigate, but they can’t find any correlating events. Over the next few days, the SIEM continues to generate new security alerts, and management is now highly concerned that they will be on the front page of the Wall Street Journal as the next ransomware victim.
As their newest security analyst, you haven’t had time to carefully review the aging SIEM deployment yet, but your gut instinct triggers an overwhelming sense of dread about the root cause of the issue.
How would you feel if I told you that this event actually occurred on July 21, 2019?
Looking at the timestamp again, "timestamp":"19-07-21 11:08:21.123"
, can you tell if the timestamp is formatted in day-month-year or year-month-day style?
This missing fact was revealed only after speaking with the application security team and reviewing current logs as they were captured in Cribl Stream. The bottom line: You’ve got to get to work correcting and fixing misconfigured timestamps.
Is this timestamp in a day-month-year or year-month-day format?
Check out these date formats by country to demonstrate how challenging timestamp recognition can be. Let’s review a few approaches where Stream can assist with correcting misconfigured timestamps.
One of the first steps of processing, when an event arrives in Stream, is to use the configured Event Breakers to determine event delineation and extract the correct timestamp. Sometimes custom event timestamps are received, and Stream needs additional configuration to properly choose the real timestamp format. Event breaking is one of the first places that the correct timestamp can be set on an event.
Additional functions such as Auto Timestamp, and Stream internal methods like , can also help, especially if time has different locations and formats within the events. You can even see event output with the data preview. Let’s walk through a few ways that Stream can help correct misconfigured timestamps.
You begin by inserting the time-challenged event as sample data, using Stream’s Data Preview.
Once you paste the event, Stream attempts to recognize and highlight the timestamp in purple.
Since the timestamp strptime format is not yet defined, Stream used the current time for the event timestamp on the left side of the event preview.
When Stream processes events in the Event Breakers, it must first locate a timestamp anchor. From there, the engine will try to do one of the following:
The closer an anchor is to the timestamp pattern, the better the performance and accuracy, especially if multiple timestamps exist within an event.
For the manually supplied option, the anchor must stop immediately before the timestamp pattern begins. Let’s define the timestamp anchor and strptime format of this event.
Set the timestamp anchor to "timestamp":"
. The anchor will then be highlighted in blue in the preview pane. Regular expression syntax is supported in the Timestamp Anchor field.
The event from 2019 was showing up in the year 2021 because no one had applied a proper locale, such as a year-month-date format, to the timestamp. This mistake will certainly cause a misconfigured time stamp
The strptime reference provides a format of %y-%m-%d %H:%M:%S.%L
that must be added to the Timestamp Format > Manual Format field to correctly process the timestamp according to the locale we expect.
Since the timestamp format is now defined, Stream extracts the proper timestamp from the event, placing the original event’s correct time on the left side of the preview.
If you find that the time of the event does not match the timestamp during a sample data preview, you can easily remedy the issue using various timestamp functions in Stream pipelines. When we resort to the functions below, let’s assume that the Event Breakers did not already correct the misconfigured timestamps.
Once you add the Auto Timestamp function to the pipeline, create a regular expression capture group that matches the time in the event, and add the strptime format of %y-%m-%d %H:%M:%S.%L
. Notice that the event timestamp on the left is now correct. The capture group in the Regex field must only extract the timestamp you will apply strptime
against, and nothing else. Here is an excellent blog on this function: Using the Auto Timestamp Function in Cribl Stream.
If you are comfortable with JavaScript expressions, the Eval function can parse the _raw event into a JSON object, and then calculate time using Stream expressions like this:
C.Time.strptime(_raw.timestamp,'%y-%m-%d %H:%M:%S.%L').getTime() / 1000
Once you identify and resolve the timestamp-related issues, you can choose to add these timestamp configurations to custom Event Breaker rulesets. We’ll cover Event Breaker rulesets in an upcoming blog.
If you want to level up your skills and learn how to fix your timestamps, try out our free Sandbox and brag about your certifications online! If you need to correct misconfigured timestamps in your own environment, process up to 1 TB of throughput per day at no cost using Cribl.Cloud.
Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.
Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari
Got one of those handy?