Cribl Search is reshaping the data search paradigm, empowering users to uncover and analyze data directly from its source. Cribl Search can easily reach out and query data already collected in Amazon S3 (or S3 compatible), Amazon Security Lake, Azure Blob, Google Cloud Storage, and more. By searching data where it lives, you can dramatically speed up your search process by avoiding the need to move data before analyzing it.  Cribl Search’s externaldata feature and lookup operators allow you to keep your lookup table up-to-date to best serve your needs This enables you to run concurrent searches, using that same up-to-date lookup table, with minimal effort. Cribl Search uses the local lookup to enrich VPC Flow Logs with security classification data from the GreyNoise API. It then sends events and IPs that GreyNoise classified as malicious to our SIEM and security analytics tool. Let’s dive into how it works.

The two main Cribl Search operators we use for the enrichment are:

• The externaldata operator 
• The lookup operator 

The externaldata Operator to collect GreyNoise Threat Intelligence

To learn more about GreyNoise Threat Intelligence APIs, visit the GreyNoise docs site.

Let’s first test the GreyNoise API independent of Cribl Search. Using this curl command, we can validate that all is working prior to Cribl Search:

curl --request GET \

--url 'https://api.greynoise.io/v2/experimental/gnql?query='last_seen:1d'&size=1000' \

--header 'accept: application/json' \

--header 'key: greynoise_enterprise_key'

Ingesting the Enrichment Data into Cribl Search

In Cribl Search, run this procedure once or twice a day to:

• Pull data from GreyNoise Threat Intelligence APIs, using Cribl Search’s externaldata operator.

• Use the project operator to extract only the IP and Classification fields.
• Use the where operator to only consider malicious IPs.
• Export the results to a local lookup table called badguys.csv.
• Use the fieldMapping option to map fields to lookup table columns.
• Use the option mode=overwrite option to update the lookup file.

The full Cribl Search command:

externaldata

[ "https://api.greynoise.io/v2/experimental/gnql?query=last_seen:1d&size=1000" ]

with( headers='{"accept":"application/json","key": "greynoise_enterprise_key"}', dataField="data")

| project classification, ip | where classification=="malicious"

| export description="Bad Guys" fieldMapping="classification:ip_status,ip:source_address" to lookup badguys mode=overwrite

• Save the search and schedule it to run once or twice per day.
  Select Actions -> Save Search As -> Schedule

Lookup Operator to Enrich Your Events with Threat Intelligence Data

Any query against this lookup will find only the malicious IPs from the last 24 hours.

• Examine the lookup file and enrichment data

Under Data -> Lookup, examine the new or updated lookup file. Notice the column names (ip_status, source_address).

• Use the lookup file to flag the IPs from the VPC flowlog data.

Cribl Search comes with out-of-the-box sample data for AWS S3 VPC Flow Logs. This data contains source address IPs (srcaddr) that might be malicious.

• Enrich this sample data with info from the badguys lookup table

Now, check the ip_status field (Field Sidebar) for value set to malicious.

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address

Or, find the malicious hits by destinations. Use this command:

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address | where ip_status=="malicious" | summarize count() by dstaddr

Forward the Data

Finally, we’ll send events and IPs that GreyNoise classified as malicious to our SIEM and security analytics tool.

The send operator forwards Cribl search results to a Cribl Stream HTTP Source, from which you can then forward and route the results to your SIEM solution.

dataset="cribl_search_sample" datatype="aws_vpcflow" | limit 10000 | lookup badguys on srcaddr=source_address | where ip_status=="malicious" | send

Wrap Up

In conclusion, Cribl Search’s externaldata and lookup operators allow you to keep your lookup table up-to-date as you see fit. This enables you to run multiple searches, using that same up-to-date lookup table, with minimal effort. The use case I walked you through here is indicative of many threat intelligence sources, and establishes an easy-to-replicate pattern for each of those types of sources. Are you ready to try it for yourself? We offer instant access to Cribl Search through Cribl.Cloud. Sign up for a free account today!

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.