ZScaler delivers a suite of well-regarded products for helping IT securely move from network infrastructure to the cloud, using zero trust principles. According to their website, they have 5,600+ customers and process 200B+ daily transactions. As organizations look to migrate to the cloud securely, they don’t always know what they don’t know, so all data is sent to their observability, security, or monitoring tools to ensure they have it all when they need it—but that’s often at odds with effectively managing budgets.

Cribl Stream can give you control over your streaming data to optimize ZScaler events without losing fidelity. A common way to reduce event size is to drop the entire event. While this is a valid approach to optimization, it is not ideal for a variety of reasons. Dropping the whole event (by event ID) can cause blind spots during a security investigation, for example. However, Cribl Stream has unique features and a powerful streaming event processing engine that allows you to clean up your events while maintaining fidelity – ensuring you’ve got the data you need to monitor and troubleshoot IT challenges and security incidents effectively.

This blog discusses how we used IF-THEN-ELSE logic to make intelligent decisions about what fields to keep and where to delete. This Fortune 500 customer did not want a universal approach to dropping duplicate fields. Instead, they asked to compare the value in Field A with the value in Field B; if they are equal, drop Field B.

If (value of A = value of B), then.
Delete B
Else
Keep B

List of identified fields names (keys) with the same potential data (values):

Here is an original (and raw) event, with sensitive data redacted:

Here is the event after Stream’s initial field extraction using our Parser function (this function’s configuration is not shown):

Here is our intelligent Eval function’s configuration (Eval adds or removes fields from events). It uses a set of JavaScript expressions to make if-then-else decisions:

A few additional cleanup items you can do along the way:

• Assign a value to the field you want to test (i.e., force a similar value to the equivalent field):

• Remove the domain name from all hostnames (saves a few bytes, since in this case, we know they’ll all have the same domain name):

• Pick up a few (relevant) contents from the user agent:

• Finally, run everything through a Serialize Function (configuration not shown) to repackage your newly formatted event into _raw (for Splunk in this case).

Finally, inspect the overall impact. As you can see, we achieved a 30.68% reduction in outbound data volume:

Summary

Cribl Stream is a purpose-built, fast, and laser-focused observability pipeline, enabling you to have fine and granular control over your events without compromising ease of use or manageability.

For ZScaler customers, Cribl Stream can help you accelerate your zero trust and cloud transformation journey, without breaking the bank.

The fastest way to get started with Cribl Stream, Edge, and Search is to try the Free Cloud Sandboxes.