Why More Flexible Cloud Service Models Are Less Expensive

The public cloud infrastructure has transformed the way businesses deploy and manage their applications, offering services that can be quickly set up and scaled according to the organization's needs.  

This article highlights the importance of bin packing for optimizing server utilization and the critical relationship between utilization and cost in server infrastructure. This will help you better understand the economic calculations of public cloud providers and make informed choices about resource allocation tradeoffs.

The economics of cloud providers: Server utilization

Server utilization is a crucial aspect that affects the cost-effectiveness of cloud providers. When a high percentage of server resources are used efficiently, it generates more value for customers, while low utilization leads to wasted capacity and increased costs. Consequently, the efficient allocation of resources is vital for cloud providers to remain competitive as it directly impacts their operating expenses and profit margins.  

In my experience, for most of the history of cloud computing, server utilization has averaged less than 50 percent, meaning revenue per server could be doubled by better utilizing resources you have already paid for. Today the best public cloud providers can achieve better than 80 percent utilization, but it is still much lower for private cloud implementations. Server capital expenditure is a sunk cost, so every additional subscriber workload that can be packed onto a server directly impacts profitability for that server.  

Bin packing: A technique for optimized resource allocation

Bin packing is a method used to improve resource allocation by arranging items of varying sizes into fixed-size containers (bins), minimizing the number of bins required. In the context of cloud infrastructure, the bin packing analogy is applied to the allocation of virtual machines, containers, or serverless functions to physical servers in a manner that maximizes server utilization.

Using bin packing algorithms, cloud providers can optimize workload placement on their infrastructure, reducing the number of idle or underutilized servers. This not only minimizes operational costs but also promotes energy efficiency, as fewer servers consume less power and generate less heat.

The power of shared resource pools

Large, shared resource pools enable cloud providers to achieve better utilization rates by taking advantage of economies of scale. By consolidating resources from multiple clients into a single, shared pool, providers can distribute workloads more effectively across their infrastructure. This approach allows for the efficient use of available resources, reducing the chances of idle or underutilized servers.

Additionally, a shared resource pool allows providers to benefit from the diverse and fluctuating resource demands of their clients. This variability in demand enables providers to allocate resources more efficiently, further enhancing server utilization and cost-effectiveness.

Affordable cloud services: Spot instances and serverless computing

High server utilization facilitated by bin packing and shared resource pools allows cloud providers to offer lower-cost services, such as spot instances and serverless computing. Spot instances are temporary, on-demand instances available at a significantly reduced cost as they utilize spare capacity in the provider's infrastructure.  

Similarly, serverless computing allows clients to run applications without provisioning or managing servers. In this model, the cloud provider dynamically allocates resources based on demand and charges clients only for the actual usage. Both spot instances and serverless compute instances are scheduled by the cloud provider automation, rather than explicitly by the user, and thus allow the cloud provider to use any available resource on any of its servers, and to move the instance if necessary to optimize utilization at runtime. These types of cloud computing models abstract the user from the details of workload placement and can thus benefit from the efficiency gains of bin packing and shared resource pools, resulting in lower provider costs that can be shared with subscribers.

Microservices and monoliths

When evaluating whether to use a monolithic executable, or to decompose as microservices, one runs directly into the utilization and bin packing dilemma. The microservice implementation consists of many smaller executable entities capable of distributed deployment. This, combined with the auto-scaling concepts inherent to microservice implementations, means that stranded capacity can be filled with the smaller container instances, directly increasing utilization of hosting nodes.

On the other hand, large monolithic implementations will typically be one instance per server, which will need to be overprovisioned which guarantees poor average utilization. There are many other complex dimensions to the microservice/monolith decision, but the microservice variant will tend to achieve higher server utilization because it is so bin-packing friendly.  

Illumio: Securing the cloud across service models

As businesses adopt public cloud infrastructure and various service models, maintaining robust security is of paramount importance. Illumio is dedicated to offering a comprehensive security solution for all popular cloud service models, enabling customers to make educated decisions about tradeoffs between infrastructure control (static resource allocation) and low cost (dynamic/abstracted resources).

Illumio Zero Trust Segmentation (ZTS) provides adaptable security measures that help organizations maintain a strong security posture, regardless of their chosen cloud service model. Whether a company opts for a traditional infrastructure-as-a-service (IaaS) approach with static resource allocation or decides to leverage cost-saving dynamic services like spot instances or serverless computing, Illumio ZTS stops the spread of ransomware and breaches through the network, protecting sensitive data and applications.

Illumio can help your organization find the perfect balance between cost-efficiency and control of cloud security. Want to learn more? Contact us today.  

‍