The Bird is the Word: Getting Up and Running Fast on Humio, by Crowdstrike

Alex Cain
Written by Alex Cain

April 12, 2022

I’ve been in the log data analytics space for years, and I have loved seeing the technology and methodologies change and evolve. One of my favorite changes has been the emergence of index-less solutions, and Humio has a great solution here. If you haven’t heard of Humio, you should check out their index-less log management solution for yourself (free up to 16 GB/day too).

In today’s blog, I want to walk you through how to set up Cribl Stream to start sending data to Humio, so you can get up and running fast. If you navigate to the destinations section in Stream, you might notice that there is no option for Humio today. In the immortal words of Douglas Adams: DON’T PANIC.

I’ll show you how.

Setting Up Humio to Receive Data From Cribl Stream

To start, let’s make sure you have a repository for this data setup in Humio. For a little context, think of Repositories as a collection of data, parsing rules, users, dashboards, searches, and alerts (if you want to learn more about it, I suggest starting here). Specifically, we need to do this step as data inputs are tied to a repository.

Once you’ve created your repository (or if you already have one), you will need to get your ingest token. Do this by clicking on the repo you want to send data to, this will take you to the search page. In my example, I’ve created a repo called “infra-metrics”.

Click on the “infra-metrics” repo and navigate to “Settings” for the repo, located on the top bar.

Once on the settings page go to the “Ingest Tokens” side tab.

We are going to create a new token called “Cribl-Stream”, without selecting an assigned parser. This is important (we’ll go into why after we set up the destination in Cribl Stream).

Once saved click on the eye icon (…or eye-con 😉 ), and copy the token.

Great! Now we have Humio set up to receive data, we just need to start sending!

Setting Up the Stream Destination

So let’s head to the Stream “destinations” page and select a “Splunk HEC” destination.

We are going to add a new “Splunk HEC” input for Humio. Feel free to name your output however you want, but we do need to change the endpoint. Instead of the standard Splunk format we are going to use the following URL structure:

If you have a custom endpoint: <URL for your Humio deployment>:443/api/v1/ingest/hec

OR

If using the community edition: https://cloud.community.humio.com:443/api/v1/ingest/hec

Then paste the ingest token we created earlier in as the “HEC Auth Token”.

Once you save it, open the configuration page back up and go to the “test” tab so we can send some sample data.

You should see the sample log go through without a hitch, and if you flip over to Humio, the events should have already shown up.

Alright, we now have the destination setup in Stream, we just need to start sending data in! The destination is ready to be used like any other destination in Stream, you just need to attach it to a route.

Parse That Data

Okay, I suppose you might have a few questions on how data will be treated by Humio if it’s being sent for Splunk. Well TL;DR; Humio tries to convert data sent to a given token based on the parser assigned to it, if you leave it blank it will try to guess.

Oh, you want the longer version?

Humio will read the fields designed for Splunk notation to Humio notation without you needing to do much of anything. But this is where the parsers kick in. If you are having trouble with the format of events in Humio, that’s where you need to start looking.

Inside your repo on Humio you will see a tab named “Parsers”, if you open that page you will see all the pre-built parsers.

If you click into any one of these parsers you will see the parsing logic or regex used for that data type.

When you leave the assigned parser empty, Humio will try to match the sourcetype (what Humio calls the “@Type”) to a parser. If your data is getting malformed, this is the best place to start looking. An easy way to make sure you have no issues with the parsers is to either create your own parser for the event in question or reformat events into JSON and set the sourcetype to “json”.

That’s all for now! Thanks for joining me on a dive into how Cribl and Humio integrate, we will be doing more with Humio over the coming weeks, so stay tuned! And remember, the bird is the word.

The fastest way to get started with Cribl Stream and Cribl Edge is to try the Free Cloud Sandboxes.

Questions about our technology? We’d love to chat with you.