Is the On-Demand Era Over?

On-Demand Instances are the most expensive of the pricing models offered by AWS, but they are by no means the only one you can leverage. AWS offers Reserved Instances at up to a 72% discount, Savings Plans at up to a 64% discount, and Spot Instances for a whopping 90% discount.

So why are so many engineers stuck paying On-Demand?

By their nature, our cloud environments are incredibly dynamic. They shift depending on our workload requirements which can vary with each new customer, feature, or a change in architecture. It’s almost impossible to know in what ways they will change.

As a result, accurately predicting our cloud usage is hard. And both RIs and SPs require you to commit far in advance, making forecasting usage extra challenging. After all, if you’re a SaaS company, how do you know how many new customers you’ll onboard next year or what the product requirements there will be in three years from today? You may also begin using popular tools like Kubernetes which will inevitably require changes in your architecture.

Despite all these factors, cost-efficient pricing models like Reserved Instances and Savings Plans do not account for this unpredictability. Depending on the family type you select, RIs require you to commit to a particular number of specific instance types 1-3 years in advance, while SPs require you to commit to spending a certain amount of compute per hour for a specific instance family 1-3 years in advance. Even for the most commitment-friendly out there, that’s way too many variables to manage to feel confident in such a long-term investment.

For this reason, RIs and SPs become a huge operational burden. Engineers will try and forecast their future usage which takes a tremendous amount of planning and calculation both mathematically and with regards to future product requirements. Yet even with all this work, accuracy is not guaranteed which results in low savings coverage, and a dependence on On-Demand Instances.

Spot Instances offer another alternative. However, their 90% discount comes at a steep price. If someone else is willing to use the same instance at the On-Demand price, your Spot Instance will be transferred to that individual with only a two-minute warning. As a result, Spot Instances are only good for workloads that are not mission-critical. Additionally, running a specific service as a Spot Instance requires engineers to develop it to be fault-tolerant. This means engineering effort will be focused on optimizing the service for costs, rather than further innovating and developing the product.