The supercloud concept promises fewer accidental architectures and more cohesive cloud deployments with better manageability. Delivering on this vision requires a mix of vendor-agnostic tooling for performance monitoring and securing data.

Core takeaways

• Enterprises developing and deploying applications to multiple public clouds are frustrated by the friction inherent between cloud operations and services and the high skills costs needed for efficient multicloud operations.
• Supercloud emerged as a concept to abstract away cloud-native primitive services into portable virtual machines with a focus on developer productivity.
• Vendors seeking to compete against incumbent public cloud providers are pushing the concept in their own private cloud offerings, but overall maturity is still low.

Recommendations

• Focus your early supercloud efforts on operations and security rather than developers. Cost-effective and performant operations, supercloud or not, still require a range of monitoring, observability, and security solutions. These often cannot be abstracted from the underlying cloud infrastructure and require specialized expertise for each cloud underpinning your supercloud infrastructure.
• While supercloud targets developers, the story is really about seamlessly moving data between cloud environments. Seek out vendor-neutral tooling to collect, process, and route data to your incumbent and insurgent tooling rather than continuing to build data silos.

What is Supercloud?

Even though the concept has existed since 2017, it’s only recently attracted attention from industry pundits and technology vendors. If you talk to the vendors, supercloud is whatever they have on the truck to sell. Pundits, like the fine folks over at Wikibon, are still formulating an acceptable definition. Industry analysts, interestingly, are largely silent on supercloud today.

We have to start somewhere, however, so let’s start at the beginning. The supercloud concept was created by researchers at Cornell University in 2017. They defined supercloud as:

…a cloud architecture that enables application migration as a service across different availability zones or cloud providers. The Supercloud provides interfaces to allocate, migrate, and terminate resources such as virtual machines and storage and presents a homogeneous network to tie these resources together.

As you can see, supercloud emphasizes application portability across cloud environments. Incorporating networking and storage for VM migration is no small feat. The Cornell researchers rely on virtual machine (VM) layering to resolve a number of portability challenges. VM layering means running your target VM inside a host VM.

The proposed benefits of supercloud are:

• Running applications across multiple clouds, as well as edge locations, provides additional application resilience and cost optimization.
• Simplified skill base for developers and users as they’re working against the abstraction layer offered by the Supercloud provider or providers.
• Application portability erodes vendor lock-in to a specific public cloud provider.

Each of the above is promising. However, supercloud also introduces new complexity into these environments.

Monitoring Architectures Get (Super)cloudy

If we’re willing to accept the core idea of seamless application portability offered by supercloud, we need to take the next step. How do we live with this architecture? We still need to monitor everything, and now there’s more to monitor: from the application VMs, to the host VMs they’re running in, and all of the native cloud services that supercloud claims to abstract away.

This monitoring expansion massively increases the cost and complexity of monitoring – more data, from more places. Costs skyrocket in traditional monitoring environments, with agents pushing all of the logs, events, metrics, and traces to a centralized system of analysis. Remember, you can abstract away the application environment, but you can’t abstract away data egress costs. Moving data from one cloud to another is costly.

Securing the Supercloud

Let’s put the monitoring challenges aside for a moment. There’s a bigger dragon hiding in those superclouds – security. Today’s security environments span on-prem, edge, and one or more clouds. There are dozens, if not hundreds, of tools used across SOCs and cloud operations teams. Each tool requires unique skills to use effectively. Shifting to a supercloud architecture changes, well, none of that. Abstracted or not, the underlying infrastructure still needs securing by highly skilled staff.

Each cloud has its own security frameworks, which work within their own domains. If the supercloud is to live up to its promise, a cohesive security framework must be created that works across environments. We’ve seen some efforts around data standardization with projects like OCSF, but nothing as yet around process standardization. That will take a level of cooperation between public and private cloud providers that we haven’t seen before, and anything that blunts their control and power over locking in users is unlikely to happen.

Supercloud Needs Open Observability

Realizing the supercloud vision requires a few critical capabilities. First, we need to control movement of our observability data, i.e. the logs, metrics, events, and traces essential to understanding application performance and security posture. Next, we must be able to integrate the raft of security tools with the right data, in the right format, at the right time. Finally, we need to reach into any component of our supercloud for investigations, troubleshooting, and exploration. Let’s break down each.

Controlling Observability Data

Think of observability data like exhaust. It’s a byproduct of the applications that produce it. As a byproduct, not all of it is useful or valuable. Yet the legacy tools we’re using today force us to ship all of it for analysis, driving up costs for data egress, storage, and platform licensing. Worse, we can typically only send that data to one destination. If there are other destinations that could use it, we’re locked into a single silo.

An open observability pipeline sits between the supercloud sources and destinations, allowing us to route data to any destination in the expected format. We can also enrich data with additional context: Which cloud did this come from? Which country does this IP address reside within? Lastly, we can choose which data to move, and when. This gives operations teams far more control and flexibility over their data.

Integrating Across Tools

As a long-time data person, I know that integration is the whole game. And not just data integration, but process integration. We can’t wait years for a supercloud security framework to emerge. We have to use what we have today. In addition to controlling your data routing and shape, an observability pipeline also acts as a strategic integration hub linking together the full range of security tools your teams are using. This is frequently referred to as a cybersecurity mesh architecture, and its core purpose is integration.

Integration is more than just linking data. It requires inherent protocol and schema knowledge. That’s what an observability pipeline brings.

Use Data Where It Resides

Open observability is more than just moving and shaping data. In the context of supercloud, we may not know the state of our applications, or even what data we can use to figure that out. Even if we could pay to move it, we don’t know what “it” is.

Open observability complements supercloud by allowing operators to seamlessly access containers, virtual machines, or even physical infrastructure to remotely debug and troubleshoot applications. By deploying an all-in-one adaptive agent alongside your application, you can access logs, interrogate running processes, and search data from across your fleet, regardless of where it might live in your supercloud implementation. This allows you to observe what’s happening across all of your infrastructure without moving all the data into high-cost analytics platforms.

Conclusion on Supercloud

The old joke that the cloud is just someone else’s computer also applies to supercloud. The difference is, with supercloud, you’re using everyone else’s computer. The challenges around monitoring, observing, and securing are compounded unless you have strong capabilities around observability data management. That’s where open observability comes in – putting you back in control.

If you want to learn more about open observability and how we build it at Cribl, check out our market-leading observability suite, available on prem, in the cloud, as part of a hybrid implementation, or in a supercloud near you.