Super Fast GeoIP Enrichment of Logs

September 18, 2019

A common use case for enriching logs is adding geographical information based on the IP address of some entity (e.g client IP, server IP etc). The needs for this enrichment vary from understanding traffic/response times/sales/etc patterns by geography to ensuring compliance. Cribl LogStream makes enriching data in motion trivial, in this post we’ll walk you through how you can add geoip information to your logs at microsecond latency ( ~5μs to be exact).

What you’ll need to get going:

1. Download Cribl LogStream (> 1.7.1) – (if you already haven’t)
2. Download MaxMind GeoLite2, make sure to choose the MaxMind Binary format, direct link GeoLite2 City
3. Untar the downloaded MaxMind database and note the path to the .mmdb file

To enrich data, first we need to have an IP address extracted – we can easily extract one by using the Regex Extract function:

… then we need to add and configure the GeoIp function:

The amount of information returned by GeoIp function is very rich:

We can then optionally, use Eval function to select only the information that we’re interested:

… and here’s how the events look like on their way out of Cribl.

If you’ve enjoyed reading this and are looking to join a kick ass engineering team drop us a line at hello@cribl.io – we’re hiring!

.
Blog
Feature Image

Cribl: Empowering Data Freedom with Open Standards and Unmatched Flexibility

Read More
.
Blog
Feature Image

Hello Vegas! Cribl @ AWS re:Invent 2024

Read More
.
Blog
Feature Image

Simplify OpenTelemetry Metrics with Cribl Edge OTLP Conversion

Read More
pattern

Try Your Own Cribl Sandbox

Experience a full version of Cribl Stream and Cribl Edge in the cloud with pre-made sources and destinations.

box

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?