Case Study

TransUnion Teams Up with Cribl Stream™ to Drive Greater Efficiency

star-round-framed
Highlights
  • TransUnion onboards and processes hundreds of sources, billions of events with Stream™.
  • The TransUnion team leverages greater speed and agility to transform and route critical data.
  • With Stream, high-volume logging doesn’t mean low-signal data.

“WE MADE A CHANGE THAT SAVED US A SIGNIFICANT AMOUNT IN LICENSING COSTS IN 30 MINUTES.”

ED BAILEY,
ENTERPRISE MONITORING AND OPERATIONS ARCHITECT, TRANSUNION

CriblCon Presentation

Routing Around the World with Cribl Stream!

Steve Koelpin, Senior Architect & Don Reilly, Senior Manager – TransUnion

Meet Your Business Goals Faster and More Affordably from Tool Evaluations, Migrations, Compliance Mandates, M&A and more
Watch Now

Share:

Download PDF
TransUnion® is a global information and insights company that makes trust possible between businesses and consumers, by ensuring that each consumer is reliably and safely represented in the marketplace.
Big Data, Bigger Expectations

Ed Bailey and his colleagues in Enterprise Logging and Analytics at TransUnion are filling a tall order; with billions of logged events a day, the kind of scale they’re working with is not commonly found in the enterprise data pipeline world. Along with scale and volume, they manage a wide, dynamic array of data sources, each of which can represent significant effort to onboard and maintain for their internal customers. The team works with Cribl to meet the challenge head on.

“We have a complex environment, challenging data, and a small team. Cribl helps us solve hard problems with greater ease.”

Ed Bailey
Enterprise Monitoring and Operations Architect

Unsurprisingly, when you’re working at such a scale, efficiency is the word of the day — and with Cribl Stream, there are many ways to say it at TransUnion:

Transform and Reroute Data on the Fly

The mission-critical job of keeping TransUnion’s customers operating safely and smoothly means Ed’s team collaborates with a lot of internal customers who use data from the same source, but in a unique way. For example, one team may need DNS metrics sent to InfluxDB, and another team needs the full DNS logs, but sent to Splunk, while another team needs data in a third format. With Stream, each department gets what they need from the original data set, where they need it, without having to install additional agents or collectors.

“Anywhere to anywhere, in any format we choose, is massive.”

David Olivas
Lead Advisor and Splunk Architect

Typically, when altering a data source’s format, you plan for some downtime while making the change, or at least a restart of agents and forwarders—but not at TransUnion:

“With Stream, we can pivot on a data change in minutes, with no need to restart anything.”

David Olivas
Lead Advisor and Splunk Architect

Before deploying Stream, the TransUnion team often set up individual data streams to achieve the desired results, and then maintained those additional streams on an ongoing basis.

“With Cribl, we can just divert existing streams to feed all the required destinations.”

David Olivas
Lead Advisor and Splunk Architect
Hot take: Not All Data is Beautiful ...or Valuable

The volume of data TransUnion engages with on a daily basis is staggering. Bailey’s team uses Stream to help ensure what’s in that data is truly useful and valuable to the teams who work with it. Recently, they were able to massively reduce the scope of high-volume logging, such as DNS and Sysmon logs teams must examine from ~1TB a day to about 150GB – a near 20x reduction.

While simultaneously sending a full-fidelity set of the data to lower-cost, longer-term storage for potential future review, Stream checks each external request against a vetted list of known trusted data in real time, enabling more than half of the requests to be filtered out as uninteresting. It then looks up the remaining traffic against known bad thread lists, this time reducing the data volume down to 150GB and providing better data.

“With Cribl, we’re able to make our operational data strategy even more effective.”

Ed Bailey
Enterprise Monitoring and Operations Architect

Windows Event Logs are notoriously bulky, but with Stream, Bailey’s team is able to strip out useless event codes and unnecessary fields, cutting the clutter and speeding the work of their internal customers.

AWS VPC Flow Logs are typically slow and expensive to deal with, but the TransUnion team runs them through Stream to suppress uninteresting or repetitive content and serves up the cleaned results in near-real time to the requesting department at a much-reduced cost.

“We made a change that saved us a significant amount in licensing costs in 30 minutes.”

Ed Bailey
Enterprise Monitoring and Operations Architect
Is it Cheating if You Just Extract the Right Answers?
At TransUnion, analysts work around the clock to review and investigate potential issues. When a customer’s business is at stake, speed is mission-critical. So when Bailey’s team learned they could accelerate time to discovery and resolution by pre-extracting just the searched-for fields from the billions of firewall events they see a day, they didn’t hesitate. They now use Stream to extract and route only what’s needed for immediate use, allowing their analysts to search for a single IP across days or weeks of firewall logs and get results in seconds rather than minutes or hours. When an interesting result is returned, the analyst can then review the full raw log for further intel. Analysts can rapidly assess and discard potential threats without needing to write a complex query to dig into the data.

“Not only does Cribl handle real-time matching on ingest, but it makes for a faster search time experience.”

Ed Bailey
Enterprise Monitoring and Operations Architect
Saying Yes— and Meaning It

The TransUnion team is excited to continue to leverage the many ways in which Cribl Stream adds to the efficiency of their streaming data operations.

“We get to say ‘yes’ more often without diverting from our team’s mission.”

David Olivas
Lead Advisor and Splunk Architect

Find out how your business can implement an observability pipeline to parse, restructure, and enrich data in flight, while cutting costs and simplifying operations.

Get Cribl, and take control of your data.

More Customer Stories

About Cribl

Cribl makes open observability a reality for today’s tech professionals. The Cribl product suite defies data gravity with radical levels of choice and control. Wherever the data comes from, wherever it needs to go, Cribl delivers the freedom and flexibility to make choices, not compromises. It’s enterprise software that doesn’t suck, enables tech professionals to do what they need to do, and gives them the ability to say “Yes.” With Cribl, companies have the power to control their data, get more out of existing investments, and shape the observability future. Founded in 2017, Cribl is a remote-first company with an office in San Francisco, CA. For more information, visit cribl.io or our LinkedIn, Twitter, or Slack community.
Pixel Mask

So you're rockin' Internet Explorer!

Classic choice. Sadly, our website is designed for all modern supported browsers like Edge, Chrome, Firefox, and Safari

Got one of those handy?