Cribl provides many different tools and constructs to help you wrangle your Metrics, Events, Logs and Trace (MELT) data. We pioneered the vendor agnostic observability pipeline, and it's been amazing watching enterprise teams use Cribl Stream to route, transform, filter, mask, aggregate and replay data to solve hard problems.
One of the often overlooked features of Cribl Stream is the Global Variable (GV). In a nutshell, a GV lets you store a value that is accessible anywhere in the worker group where you create it. But burrow into the details some, and you'll see that there's so much more to it than retrieving a simple static value.
In this post, we'll cover the basics of GVs, three relatively simple use cases for them, and a bonus round of using one to help with quota enforcement in a SIEM.
Basics
Our docs cover what you need to know to get started using Global Variables, including some nifty examples. Please read them! For the impatient: Global Variables are stores of values you can reference elsewhere in Cribl. They are found under the Processing -> Knowledge menu in a Worker Group context. By default Cribl includes a few sample GVs you can scope out to get a feel. You'll notice quickly there are different types of GVs.
String, Number and Boolean are pretty simple. Just static values. You reference these anywhere you can use JS with C.vars.varname.
Encrypted String allows you to store a string that, when written to disk in yaml files, is encrypted. Note: You'll still be able to retrieve when logged into the Cribl interface. Reference it the same as above, C.vars.varname.
Arrays and Objects are getting more interesting as you can define more complex variables. You could define a list of hostnames or IPs, then refer to that list in filters and pipelines. Or a dictionary of key value pairs in an object as another way to do lookups. Reference these as you would any other object or array. C.vars.varname[0] or C.vars.varname.nested.
Expressions turn it up to 11. You can use JS statements to define something more complex. But most importantly you can pass arguments into the expression to use in calculating the output. Call it with the arguments in parentheses: C.vars.varname(arg1,arg2).
Use Case 1: Simplify Complex Filters in Routes
I love Cribl Stream's routing table. Being able to send logs where we need to with complete filtering control at the event level is a game changer. The struggle I often see in the wild is controlling those filtering rules. They can get complicated, with mixed booleans, patterns, lookups… oy! As they get more complex it's tougher and tougher to see at a glance, or even several glances, how all the rules interact with each other. I'm talking about the intent of each rule, not necessarily the inner workings.
Picture having many routes filtering on 3 qualifiers: the source field, a regex pattern match, and a lookup of an IP address. Scanning through rules that have these 3 qualifiers in different combinations would be error prone, difficult to see which rules are taking precedence, and how events will react.
source.endsWith('palo.log') && /[^0-9]\s\w{6,}/.test(_raw) && !(C.Lookup('coolfile.csv','keyfield').match('event_field'))
With a series of Global Variable expression definitions, you could reduce each of those complex statements to something repeatable, abstracted, and easy to grok at a glance.
C.vars.Is_Logtype_Alpha(source,_raw,src_ip)
This would call an expression GV defined with the same expression as above. Importantly list source, _raw, and src_ip as the arguments.
By using GVs, you can see at a glance what the intent of the admin was. For sure, you still need to check on the validity of the expressions! But once you do, throw it in a GV where a) it's easier to see the intent; and b) less likely to be accidentally altered.
Use Case 2: Regex
Complex regex can be a pain to manage. While Cribl does offer a regex library, you cannot reference the regexes there in JS expressions. Using GVs we can get around that, and make referencing common and/or complex regex simple. And if you make reference to this complex regex in multiple places, you only need to update the one to impact all of them.
For example, I could use this regex everywhere I need to test if a string could be a MAC address, and just hope the people looking at it know that.
/([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})/.gm.test(field_name)
Or, I could encapsulate the regex into an expression:
Now the call to test for a MAC address is more easily reusable and much clearer:
C.vars.isMACaddr(field_name)
Use Case 3: Flag to Enable Features, Routes, etc
Many organizations have maintenance cycles or other periods of activity when all the log data is not needed in the (mind-blowingly) expensive SIEM. During these times, it would be a big help to have a simple process to switch the flow of log data to another place—maybe just to an object store or to a lower-cost alternative analysis tier.
We can do this easily with a GV.
Define a GV called detourTime.
Make it a boolean type, and set it to either true or false depending on an active maintenance window.
Define a route at the top of your routing table with a filter of
C.vars.detourTimeFlag the route as Final.
Alternatively, you could create an Output router for each destination you want this switch to be active on. Make the first rule filter C.vars.detourTime, point to your cheaper alternative destination, and mark it Final. Make the second route point to your normal destination with a filter of true.
And yet another option would be to use a GV that specifies the literal name of the destination. You can then use the Output Expression option in the data routes table.
Regardless of how you reference the GV, now you can toggle it (and commit & deploy) to immediately impact where your data is flowing with precision and intentionality. Notably a GV change only causes a reload, not a restart, when you deploy. It's instant.
Extended Example: GV to Enforce Quotas
Let's take that previous example use case, and add some automation nuggets to control SIEM quotas with an emergency shutoff valve.
For this example we only want 1 TB per day to land in $Expensive$IEM. We'd like an automated way to watch how much data we're sending out. Once we exceed that 1 TB number, we need to auto-toggle a GV to stop delivery to $Expensive$IEM, but continue delivering to the data lake for long term retention, and maybe investigate another day.
A little detour first. We need to get the metrics.You can either send internal metrics to your analysis tier and start there, or use the API as I'm doing here. (See my previous blog on the API: Part 1 and Part 2.) Assuming we have the login part straightened out, we need to POST something like this:
POST https://your_cribl_cloud_leader/api/v1/system/metrics/query
Content-Type: application/json
Accept: application/json
Authorization: Bearer your_bearer_token
{
"where":"output == \"webhook:prod_siem\"",
"aggs":{"aggregations":[
"sum(\"total.out_events\").as(\"events\")",
"sum(\"total.out_bytes\").as(\"bytes\")"
],
"cumulative":true},
"earliest":"-0d@d"
}
You'll need to list the output ID yourself. The format is the type of output, colon, name of the input. You can find them in the monitoring console under Data, Destinations. And you'll need to update the earliest value as needed. In this case, I just listed this morning at midnight. But that's in UTC. Your requirements may dictate this be changed. Other options are an epoch time in milliseconds, or a text string, eg 2025-04-14T00:00:00:000Z
Once you do it correctly, the result of the query will look like this:
{
"results": [
{
"events": 10387444,
"bytes": 4167462674
}
], ...snip...
}
With that data in hand, now we can make a decision to turn off the spigot. If results[0].bytes / 1_000_000_000 > quota_in_GBs, then we need to fire …
The GV update action. This is another API call, assuming your GV is named quotaDetour. Notice there are two places you need the GV name.
PATCH https://your_cribl_cloud_leader/api/v1/m/your_wg/lib/vars/quotaDetour
Content-Type: application/json
Accept: */*
Authorization: Bearer your_bearer_token
{"type":"boolean","value":"false","id":"quotaDetour"}
Now we need to commit and deploy this change – and this change only:
POST https://your_cribl_cloud_leader/api/v1/version/commit
content-type: application/json
accept: application/json
Authorization: Bearer your_bearer_token
{
"message": "quota control change",
"group": "your_wg",
"files": ["groups/your_wg/local/cribl/vars.yml"]
}
The response from this action will be JSON payload including a commit ID at the path response.items[0].commit. We need it for the deploy action:
PATCH https://your_cribl_cloud_leader/api/v1/master/groups/your_wg/deploy
content-type: application/json
accept: application/json
Authorization: Bearer your_bearer_token
{
"version": "commit_id"
}
Changing global variables is not something that requires worker processes to restart. They simply reload the values, so this will process very quickly without any interruption. Now in the destination for your SIEM, you can use routes, pipeline rules, or an Output Router with reference to this variable:
For this Output Router, If quotaDetour is false, we match the rule, and deliver to the webhook destination. I have final checked just for completeness. (See below)
If quotaDetour is true, the first rule does not match, and in this case there are no more, so the event is dropped. You could list an alternative destination in a 2nd rule, like maybe an OSS instance of Elastic. (Which is why I chose the Output Router method here, and have the rule marked Final.)
How you use the variable to trigger the alternate routing and/or dropping is up to you. We are nothing if not extremely flexible.
Finally, don't forget to reset at midnight to return to normal.
I have compiled this rough sequence of steps into a python script. I encourage you to TEST IT OUT and see how it works. Please be cautious about putting it into production. It is provided with no warranty – simply a starting point for you to build from.
Conclusion
We've detailed a few use cases for Global Variables (GVs) in Cribl Stream, highlighting their versatility beyond basic value storage. GVs can simplify complex filters, manage regex expressions, act as flags to control data flow, and even be used to help automate traffic flow. How will you use GVs? Come on over to the Cribl Community to share, ask questions, or just hang out.
