Cribl Search Threat Hunting

Using Cribl Search to Aid in Threat Hunting by Enriching Data in Motion

October 31, 2023
Written by
Raanan Dagan's Image

During his many years at Cribl, Splunk, Cloudera, and Oracle he was part of multiple impl... Read Moreementations of security, analytics, cloud, open-source, and IT use cases as well as big data and data lake projects in complex environments. Raanan is a global resource with 30 years of experience building large data clusters. He has helped thousands of customers, including some who ingest several hundred terabytes per day and store multiple petabytes of data. Read Less

Categories: Cribl Search, Engineering

Cribl Search is reshaping the data search paradigm, empowering users to uncover and analyze data directly from its source. Cribl Search can easily reach out and query data already collected in Amazon S3 (or S3 compatible), Amazon Security Lake, Azure Blob, Google Cloud Storage, and more. By searching data where it lives, you can dramatically speed up your search process by avoiding the need to move data before analyzing it.  Cribl Search’s externaldata feature and lookup operators allow you to keep your lookup table up-to-date to best serve your needs This enables you to run concurrent searches, using that same up-to-date lookup table, with minimal effort. Cribl Search uses the local lookup to enrich VPC Flow Logs with security classification data from the GreyNoise API. It then sends events and IPs that GreyNoise classified as malicious to our SIEM and security analytics tool. Let’s dive into how it works.

A screenshot of a computer Description automatically generated

The two main Cribl Search operators we use for the enrichment are:

The externaldata Operator to collect GreyNoise Threat Intelligence

To learn more about GreyNoise Threat Intelligence APIs, visit the GreyNoise docs site.

Let’s first test the GreyNoise API independent of Cribl Search. Using this curl command, we can validate that all is working prior to Cribl Search:

curl --request GET \

--url 'https://api.greynoise.io/v2/experimental/gnql?query='last_seen:1d'&size=1000' \

--header 'accept: application/json' \

--header 'key: greynoise_enterprise_key'

Ingesting the Enrichment Data into Cribl Search

In Cribl Search, run this procedure once or twice a day to:

A screenshot of a computer Description automatically generated

  • Use the project operator to extract only the IP and Classification fields.
  • Use the where operator to only consider malicious IPs.
  • Export the results to a local lookup table called badguys.csv.
  • Use the fieldMapping option to map fields to lookup table columns.
  • Use the option mode=overwrite option to update the lookup file.

The full Cribl Search command:

externaldata

[ "https://api.greynoise.io/v2/experimental/gnql?query=last_seen:1d&size=1000" ]

with( headers='{"accept":"application/json","key": "greynoise_enterprise_key"}', dataField="data")

| project classification, ip | where classification=="malicious"

| export description="Bad Guys" fieldMapping="classification:ip_status,ip:source_address" to lookup badguys mode=overwrite

 

  • Save the search and schedule it to run once or twice per day.
    Select Actions -> Save Search As -> Schedule

Lookup Operator to Enrich Your Events with Threat Intelligence Data

Any query against this lookup will find only the malicious IPs from the last 24 hours.

  • Examine the lookup file and enrichment data

Under Data -> Lookup, examine the new or updated lookup file. Notice the column names (ip_status, source_address).

A screenshot of a computer Description automatically generated

  • Use the lookup file to flag the IPs from the VPC flowlog data.

Cribl Search comes with out-of-the-box sample data for AWS S3 VPC Flow Logs. This data contains source address IPs (srcaddr) that might be malicious.

  • Enrich this sample data with info from the badguys lookup table

Now, check the ip_status field (Field Sidebar) for value set to malicious.

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address

Or, find the malicious hits by destinations. Use this command:

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address | where ip_status=="malicious" | summarize count() by dstaddr

A screenshot of a graph Description automatically generated

Forward the Data

Finally, we’ll send events and IPs that GreyNoise classified as malicious to our SIEM and security analytics tool.

The send operator forwards Cribl search results to a Cribl Stream HTTP Source, from which you can then forward and route the results to your SIEM solution.

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address | where ip_status=="malicious" | send

Wrap Up

In conclusion, Cribl Search’s externaldata and lookup operators allow you to keep your lookup table up-to-date as you see fit. This enables you to run multiple searches, using that same up-to-date lookup table, with minimal effort. The use case I walked you through here is indicative of many threat intelligence sources, and establishes an easy-to-replicate pattern for each of those types of sources. Are you ready to try it for yourself? We offer instant access to Cribl Search through Cribl.Cloud. Sign up for a free account today!

 


 

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.

.
Blog
Feature Image

Leveraging AWS Private Image Build for a Compliant Cribl Deployment

Read More
.
Blog
Feature Image

Cribl: Empowering Data Freedom with Open Standards and Unmatched Flexibility

Read More
.
Blog
Feature Image

Hello Vegas! Cribl @ AWS re:Invent 2024

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?