Mainframes might seem like relics of a bygone era to many of us in 2024, but the truth, however, is far from that. Despite their reputation as ancient behemoths—and frequent targets of jokes—mainframes continue to be vital powerhouses driving the global economy. Their capability to process billions of transactions daily, including the majority of credit card transactions, underscores their enduring significance. These transactions proceed uninterrupted, even in the face of natural disasters like earthquakes, showcasing the mainframe’s unparalleled reliability. With support for up to 40TB of memory, mainframes were the original data engine of Big Data, handling vast datasets more cost-effectively than many of today’s cloud-based solutions.

It’s not surprising then, that 71% of Fortune 500 companies still rely on mainframes. However, with great processing power comes the inevitable generation of vast amounts of log data. The challenge for many IT professionals today is not just understanding the value of mainframes but effectively managing and making sense of the log data they produce. Fortunately, advancements have kept pace with the times, offering modern log collection and analysis solutions.

Starting with Syslog

The first stop in the journey of mainframe log management should be the built-in syslog service of the mainframe. This tool logs activities on the operating system and, to some extent, within applications. Leveraging syslog is cost-effective as it incurs no additional expenses and is a familiar technology to IT Operations and Security Operations teams, thanks to its use of standard TCP syslog. This compatibility with existing systems simplifies the integration with log management tools like Cribl Stream, allowing for easier consumption and analysis of the data.

However, the sheer volume of data generated can be overwhelming. To mitigate this, using features such as data reduction and filtering within log management tools is essential to manage the flood of information effectively without compromising on the insights needed for decision-making. In addition, Cribl Stream’s enrichment features make it easy to enrich mainframe events using lookup files to help make events that are famously sparse more actionable for operations teams.

As admin sought greater visibility into Mainframes to bridge the many silos of data, several companies have released commercial tools that seek to expose all the information a mainframe can generate to commercial logging and observability platforms.

Exploring Commercial Tools

For those seeking more comprehensive coverage, commercial tools like IronStream from Precisely offer a deeper dive into mainframe data. These tools can provide access to a broader range of data across the mainframe stack, presenting it in formats that are more user-friendly. While this option offers the most extensive insights, it comes at a higher cost, proportional to the mainframe’s capacity. It’s a premium solution for those who need the most detailed information and are willing to invest in it.

There are also other tools in the market, such as IBM Z CDP, PowerExchange Logger from Informatica and Mainframe Explorer from Micro Focus, along with APM tools from Dynatrace and AppDynamics designed specifically for mainframes. While these options are available, they may not offer the same level of utility or ease of use as more established solutions like IronStream.

Building Custom Solutions

When cost constraints are a significant concern, or when available commercial tools do not meet specific needs, building a custom solution is a viable path. This approach involves creating a bespoke system where the mainframe directs event logs to a custom queue, from which the logs are then extracted and analyzed. While this method offers flexibility and cost savings, it comes with its own set of challenges, including limited formatting options and the inherent complexities of developing and maintaining custom software. This is a challenge for only the desperate and most committed. Learning the ins and outs of FFI and FFR file formats is a memorable experience best avoided if possible.

Conclusion

Mainframes remain at the heart of global business operations, processing critical transactions and managing vast datasets with unrivaled reliability. As modern IT professionals, understanding how to effectively collect and analyze mainframe log data is crucial. Whether through built-in syslog services, commercial tools, or custom-built solutions, there are numerous paths to harnessing the power of mainframe data. The choice depends on the specific needs, resources, and goals of your organization, but one thing is clear: ignoring the potential of mainframes is no longer an option in today’s data-driven world.

Evidence

10 Mainframe Statistics That May Surprise You

Mainframes Are A Critical Part Of Modern IT Strategies

Cribl, the Data Engine for IT and Security, empowers organizations to transform their data strategy. Customers use Cribl’s suite of products to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.

We offer free training, certifications, and a free tier across our products. Our community Slack features Cribl engineers, partners, and customers who can answer your questions as you get started and continue to build and evolve. We also offer a variety of hands-on Sandboxes for those interested in how companies globally leverage our products for their data challenges.