The Scene
Like most sizable, established enterprises, Autodesk’s data infrastructure is extensive and complex. As the organization has grown, so has the volume and content of its logging data. Each business unit generates a vast quantity of operational data, and each business unit derives market-specific context and value from the data it generates.
As a thriving, diverse, global enterprise, Autodesk also produces a profusion of different types of data, and makes heavy use of multiple logging and SIEM (Security Information and Event Management) platforms to enable its organizations to leverage and analyze the escalating flow of machine-generated information.
“We’ve been running an Enterprise SIEM for a decade or more, and other major log manage-ment has been in the mix for a long time as well.”
Jacob Gorney
Cloud Architect
While these tools provide Autodesk’s analysts with the structure they need to dig into their data and use it to make strategic and practical decisions, significant resources and attention are also required to keep them running well.
“Over the years, we’ve hair-pinned some stuff on the back end, resorted to duct tape and glue at times to keep things working the way our (internal) customers expected them to work.”
Jacob Gorney
Cloud Architect
Logging Data Analytics Engineering Manager Sudha Kanupuru explains that her team used to spend all their working hours supporting Autodesk’s data infrastructure to ensure their internal customers had the data they needed to move the business forward.
“We were heavily focused on operational work and always trying to solve the latest problem.”
Sudha Kanupuru
DevOps Engineering Manager
When data flow suddenly spikes, because something unusual happens – that is when real-time access to log data is most critical, and licensing models for logging services is challenging.
“When there’s a sudden influx of data and we exceed our license, our SIEM can block access. Logging system nodes crash when they reach capacity. Our alerts help us catch these situations.”
Sudha Kanupuru
DevOps Engineering Manager
Sudha and Jacob both envisioned a future moving away from the operational support space, and away from constantly configuring custom inputs, props, and transforms; worrying about managing data flow; and checking to ensure critical infrastructure and security logs were flowing in and accessible.
Enter Cribl
Concurrently, Jason Smathers, Senior Manager of the Cloud Architecture & Insights team, heard about a product called Cribl Stream, an observability pipeline. Jason’s team is responsible for logging practice, general data governance and oversights, policies, architecture, and spend for AWS, and other related areas.
“What I had heard about Cribl’s impact and how it helps you gain control over your data and ever-increasing logging costs resonated with me.”
Jason Smathers
Senior Manager, Cloud Architecture & Insights
The timing was also right. To streamline development and accelerate time-to-market, Autodesk’s technical leadership made the decision to steadily expand its use of cloud-based services and decrease its datacenter footprint. Jason wanted to see the entire Autodesk enterprise use the cloud migration of its business data sources as an opportunity to break free from the complexity of accumulated technical debt in associated data pipelines.
He didn’t have to suggest it twice. Jacob and Sudha saw their chance to eliminate excess pipeline complexity, and ran with it. Working with Cribl, they collected the information they needed and made a plan, drawing on deep industry expertise from the Cribl team.
“We were so focused on keeping the stack running that we sought help on specifics, and Cribl’s team helped with institutional knowledge...Cribl’s support has been incomparable in terms of helpfulness.”
Jacob Gorney
Cloud Architect
Once the plan was assembled, the cut-over to using Stream was quick and pain-free.
“We did it all in less than half a day.”
Jacob Gorney
Cloud Architect
And Now?
Not only is Autodesk’s data pipeline infrastructure more modern, uncomplicated, and resilient, but costs associated with the analytics tools in use are markedly lower as well.
“Lots of data was being duplicated in our logging platforms, and Stream allows us to easily detect when this happens. We were able to reduce extra ingestion by 93.1 percent an internal target we set for ourselves by just doing the easy stuff!”
Sudha Kanupuru
DevOps Engineering Manager
But it’s not just about saving money: Sudha is still uncovering opportunities to prioritize more valuable data. The organizations her team serves throughout Autodesk can now use Cribl to refine data ingestion to focus on the events that matter most to the business.
“The goal is to optimize the data. Our priority is to make the data more valuable, mark it up, add more--it’s not just about cost-cutting, it’s about getting more context to make better decisions.”
Sudha Kanupuru
DevOps Engineering Manager
Find out how your business can implement an observability pipeline to parse, restructure, and enrich data in flight, while cutting costs and simplifying operations.
Get Cribl, and take control of your data.
