Are you looking to correct misconfigured timestamps from logs in your Security Incident and Event Management tool? Let’s say you’ve just received a critical security alert from the SIEM tool regarding a brute-force attack that occurred on July 19, 2021, at 11:08AM. The cryptic event looks like this:

The Incident Response team was dispatched to investigate, but they can’t find any correlating events. Over the next few days, the SIEM continues to generate new security alerts, and management is now highly concerned that they will be on the front page of the Wall Street Journal as the next ransomware victim.

As their newest security analyst, you haven’t had time to carefully review the aging SIEM deployment yet, but your gut instinct triggers an overwhelming sense of dread about the root cause of the issue.

How would you feel if I told you that this event actually occurred on July 21, 2019?

Looking at the timestamp again, "timestamp":"19-07-21 11:08:21.123", can you tell if the timestamp is formatted in day-month-year or year-month-day style?

This missing fact was revealed only after speaking with the application security team and reviewing current logs as they were captured in Cribl Stream. The bottom line: You’ve got to get to work correcting and fixing misconfigured timestamps.

Is this timestamp in a day-month-year or year-month-day format?

Check out these date formats by country to demonstrate how challenging timestamp recognition can be. Let’s review a few approaches where Stream can assist with correcting misconfigured timestamps.

One of the first steps of processing, when an event arrives in Stream, is to use the configured Event Breakers to determine event delineation and extract the correct timestamp. Sometimes custom event timestamps are received, and Stream needs additional configuration to properly choose the real timestamp format. Event breaking is one of the first places that the correct timestamp can be set on an event.

Additional functions such as Auto Timestamp, and Stream internal methods like , can also help, especially if time has different locations and formats within the events. You can even see event output with the data preview. Let’s walk through a few ways that Stream can help correct misconfigured timestamps.

Data Preview

You begin by inserting the time-challenged event as sample data, using Stream’s Data Preview.

Pasting Events

Once you paste the event, Stream attempts to recognize and highlight the timestamp in purple.

Default Timestamps

Since the timestamp strptime format is not yet defined, Stream used the current time for the event timestamp on the left side of the event preview.

Timestamp Processing

When Stream processes events in the Event Breakers, it must first locate a timestamp anchor. From there, the engine will try to do one of the following:

• Scan (up to a configurable depth) into the event and autotimestamp it
• Timestamp using a manually supplied strptime format
• Timestamp the event with the current time

The closer an anchor is to the timestamp pattern, the better the performance and accuracy, especially if multiple timestamps exist within an event.

For the manually supplied option, the anchor must stop immediately before the timestamp pattern begins. Let’s define the timestamp anchor and strptime format of this event.

Timestamp Anchors

Set the timestamp anchor to "timestamp":". The anchor will then be highlighted in blue in the preview pane. Regular expression syntax is supported in the Timestamp Anchor field.

Timestamp Format

The event from 2019 was showing up in the year 2021 because no one had applied a proper locale, such as a year-month-date format, to the timestamp. This mistake will certainly cause a misconfigured time stamp

The strptime reference provides a format of %y-%m-%d %H:%M:%S.%L that must be added to the Timestamp Format > Manual Format field to correctly process the timestamp according to the locale we expect.

Timestamp Recognition

Since the timestamp format is now defined, Stream extracts the proper timestamp from the event, placing the original event’s correct time on the left side of the preview.

Fixing Time in the Pipeline

If you find that the time of the event does not match the timestamp during a sample data preview, you can easily remedy the issue using various timestamp functions in Stream pipelines. When we resort to the functions below, let’s assume that the Event Breakers did not already correct the misconfigured timestamps.

The Auto Timestamp Function

Once you add the Auto Timestamp function to the pipeline, create a regular expression capture group that matches the time in the event, and add the strptime format of %y-%m-%d %H:%M:%S.%L. Notice that the event timestamp on the left is now correct. The capture group in the Regex field must only extract the timestamp you will apply strptime against, and nothing else. Here is an excellent blog on this function: Using the Auto Timestamp Function in Cribl Stream.

The Eval Function

If you are comfortable with JavaScript expressions, the Eval function can parse the _raw event into a JSON object, and then calculate time using Stream expressions like this:

C.Time.strptime(_raw.timestamp,'%y-%m-%d %H:%M:%S.%L').getTime() / 1000

Once you identify and resolve the timestamp-related issues, you can choose to add these timestamp configurations to custom Event Breaker rulesets. We’ll cover Event Breaker rulesets in an upcoming blog.

Wrap Up on Correcting Misconfigured Timestamps

If you want to level up your skills and learn how to fix your timestamps, try out our free Sandbox and brag about your certifications online! If you need to correct misconfigured timestamps in your own environment, process up to 1 TB of throughput per day at no cost using Cribl.Cloud.