How to Define and Create a Realistic Cloud Budget

All large projects need clear and thoughtful planning. And a big part of planning involves figuring out how much you’ll need to spend and where your money will go, i.e. a budget.When it comes to creating a realistic cloud budget, things get complex very fast.

In the good old days, IT projects involved a relatively small number of moving parts. The big-ticket items included the purchase and maintenance of servers and networking equipment along with ongoing costs like power, cooling, and facilities security. Building reliable and meaningful budgets certainly may not have been easy, but there wasn’t a lot of mystery involved.

All that’s changed with the shift to cloud platform hosting. You won’t be purchasing and installing $100,000 server room racks any more, but working with the virtual tools that have replaced them can sometimes feel less intuitive.

Deploying a complex, multi-tiered application in the cloud could leave you with ongoing charges for virtual servers, containers, serverless functions, managed databases, network bandwidth, firewall appliances, data storage, and monitoring software. Consider how this is also configured to scale up or down in response to fluctuating demand or system failure, all while being securely replicated for high availability.

To top it all, AWS is constantly updating both the features and pricing for their existing services. They introduce dozens of new services each year, affecting not only what you’ll pay over time (often to your advantage), but how you do things, too.

With so many moving parts in play, the clarity of a solid budget is absolutely critical. Read on to see what’s involved in this process.

The Challenges of Building a Realistic Cloud Budget

So, just what is it that makes the cloud so complicated? A generation ago, an average mid-sized company might have maintained data centers with fleets of servers built with a small handful of hardware architectures. Database servers might have larger drives, while application servers would be given more memory. But for all practical purposes, the available options were few, and the mix would remain largely stable for years at a time.

How does that work in the AWS cloud? There are currently more than 200 distinct services, each offering a unique set of capabilities that can be consumed on their own or, more often, as part of a larger deployment stack. And each service uses its own unique billing model.

Virtual server types? There are, at the moment, more than 40 instance type families available for the EC2 service, each one optimized for different use cases. And most of these families include at least a dozen specific instance types, with more being added each month. And if that’s not enough, their costs change frequently, like the rest of AWS services—they become cheaper for existing instance classes. Meanwhile newer, better instance classes are introduced with higher price ranges.

Also, applications needs change all the time. New products, new features, customer on-boarding, customer churn—all affect the size of your cloud infrastructure. For example, adding a new microservice to your application stack may look simple from a solution architecture perspective, but it may translate into creating another node for the Kubernetes cluster. How do you define a budget if needs constantly change?

To maintain high service availability, you’ll often configure your services to automatically scale up and down to meet changing demand. But even normal utilization changes due to, say, your customers’ seasonal shopping patterns, can dramatically impact your billing rates for multiple services.

Cloud budgets are clearly not for the faint of heart.

Cloud Budgets: What to Consider First

But it’s not all doom and gloom. After all, millions of businesses create a cloud budget, and they need to start somewhere.

In order to get started, one thing you need to have handy from the get-go is the organization’s revenue projections or goals. This is to ensure your budgeted infrastructure costs are realistic. You can have a budget of any figure, but it needs to be realistic in the sense that it should fit within the overall IT budget, and it should be less than the projected revenue to ensure the company makes a profit.

If you are starting with your cloud journey, understanding your deployment strategy can help in having a high-level understanding of costs involved. For example, if you are doing a lift-and-shift of an application from your on-premise environment, the architecture would most probably be simpler in the cloud, and be less costly. If you are doing a complete re-architecture, it will probably need more services, which will cost more.

When moving an existing on-premises stack to the cloud—or even when you’re just comparing your options—it can be helpful to create parallel budgets that can be viewed side-by-side. The concept of “parallel budgets” will also apply to the multiple infrastructure environments that are often used for larger projects: one for the development team, a staging environment for final testing, and then the actual production environment. Each one comes with its own costs as these environments are naturally different in size.

Similarly, when you are starting out with cloud, whether the on-premise network will remain in operation for the foreseeable future is another factor to consider. Having footprints in both environments will increase the total cost of ownership (TCO), and your budget will need to minimize that as much as possible.

It’s also important to differentiate between your capital expenses (capex) and operating expenses (opex). Capex is the money you spend up front on compute equipment and physical infrastructure. Opex, by contrast, represents ongoing month-by-month expenses. Pure cloud deployments, since they’re billed incrementally for actual resource use, will qualify as opex. But the infrastructure you might run in your own data center, perhaps as part of a hybrid deployment, will require up-front capex investments. Both, however, are key elements for calculating the TCO of your application. And neither can be ignored.

You should also understand the difference between managed and unmanaged services. Managed services include the AWS Relational Database Service (RDS) and Elastic Beanstalk. They’re managed in the sense that you only need to configure some settings and then add your code or data. AWS manages the underlying infrastructure invisibly. Unmanaged services include EC2 instances, where you handle all the update and configuration, at least on the software level.

Finally, a complete budget will also take into account any cost-saving elements you may already have in place like Reserved Instances and Savings Plans. Will you need servers running at full capacity 24 hours each day? Then you’ll probably get the most bang for your buck by purchasing EC2 Reserved Instances. Demand that’s intermittent and rarely stretches for more than a few hours at a time, though, might be best met using On-Demand Instances in order to avoid underutilizing commitments. In-house data processing that’s loosely coupled enough to withstand sudden outages is a good match for Spot Instances. And brief bursts of programming code can often be served most efficiently from Amazon’s “serverless” service, Lambda.

The big thing, as we’ve already seen, is understanding what the best practice is for running your workloads and what they are supposed to accomplish. And the bottom line is that the amount of money you spend should accurately and predictably make sense in the context of your organization’s larger goals.

To create a useful budget you’ll, therefore, need to understand:

• Your application and the larger business model that’s behind it.
• What is the total cost of a customer? How many cloud resources is one customer consuming?
• How the “cloud approach to IT” will work for your specific application.
• Whether you’ll be integrating third-party cloud or on-premises resources into your infrastructure and whether that means you’ll need to purchase premium network connectivity.
• Which compute, data storage, database, networking, firewall, and serverless services you’ll use for your specific application.
• How each of those services is billed.
• How much of each service you’ll consume over time.

Cloud Budgets: General Guidelines to Know

Once you have done the groundwork, developing a realistic cloud budget will still take some upfront preparation.

First of all, involve the teams that will be directly affected by the budget. This includes application, DevOps, SecOps, operations, and finance. Try to make it a team effort, because budgeting is not a one-man, nor even a one-team, effort.

Second, devise and implement a solid tagging strategy for all your resources. An inch in this case will take you a few miles. Anyone can tag their resources, but what needs to be done here is to agree upon a tagging style and implement that as part of your infrastructure provisioning automation.

Tagging resources effectively will provide you with granular visibility into how much each application\service costs.

Third, you may want to break down the AWS accounts into more manageable “chunks” using a multi-account strategy. For example, using an account for the development workload and an account for production will help you separate the development budget from the operational budget and plan ahead.

For AWS cloud platform, there are many powerful financial tools already available out of the box. As complicated as AWS billing can get, the company works hard to help you visualize the charges that are coming.

The AWS Pricing Calculator, for instance, lets you enter precise details of the AWS resources you’re actually planning to use. The calculator is designed to be deeply integrated with all the major AWS services. This way, the estimates you get will closely reflect real-world billing conditions. There’s also the Simple Monthly Calculator which the Pricing Calculator is supposed to replace.

Once you begin consuming AWS services, there’s a full range of tools to help you visualize what you’re running and how much it will cost you. Descriptions of the full set of Amazon’s cost management services are nicely collected on this page. Those tools include:

• AWS Cost and Usage Report to view detailed ongoing and historical reports of your spending.
• Billing & Cost Management Dashboard, a central hub for creating and managing budgets, spend history, and usage reports.
• AWS Cost Categories, which lets you organize your active resources by account, service, charge type or, in conjunction with a configured resource tagging strategy, by tag.
• AWS Application Cost Profiler for breaking down your spending on resources by individual users. Organizations large enough to have multiple teams independently deploying resources will be better positioned to watch what’s going on.

AWS Control Tower is designed to let you view and control permissions and access for those with multiple AWS accounts (via AWS Organizations) from a single browser-based page.

And don’t forget to leverage your own internal tools. As popular wisdom would have it, you can never have too many spreadsheets. Create a parameterized plan in a spreadsheet describing multiple usage scenarios and share it with all the project stakeholders, including representatives from your operations, security, development, management and finance teams. This again ties with your approach of involving different teams from the beginning.

Above all—keep an open mind, incorporate relevant data from anywhere, and keep refining your plans until you hit perfection.

Implementing Governance

One way or another, any decision connected to a cloud deployment is going to impact spending. To make sure that your project never spins out of control, everything that happens has to be noticed by someone with responsibility. In other words, there needs to be a governance element to monitor and enforce the budget, which should incorporate administration oversight through both automated and manual audits.

AWS CloudWatch can be set to monitor an account for unexpected billing events. Billing alarms can be triggered when resources begin generating costs at rates above preset limits.

You can limit your risk of rogue, shadow IT deployments—where individuals or even teams in your organization bypass management to quietly launch their own resources—using IAM users and groups. Users and groups can also be configured with roles and policies to permit individuals or teams just enough access to do their work.

Final Words

Complete and reliable budgets for cloud deployments don’t happen overnight. You need to ensure you’re getting all the technical details right in a way that supports your organization’s goals rather than confounds them. That’s a process that requires time, careful thought, and input from all the experts you can reach.

Start small, perhaps budgeting for a single environment, application, or cost-centre. Then use the knowledge and experience you gain to build the overall budget.

Once you create your cloud budget, you’ll need to put together the right processes and procedures that will enable you to stick to it in the long term.

Luckily, Zesty can help with that. Chat with one of our cloud experts to learn how Zesty automates cloud savings with zero engineering effort.

Good luck!