CIO/CTO Corner: overspend in the cloud, part 1

Introduction

According to the latest reports, 94% of enterprises are overspending in the cloud while the long-term trend of cloud migrations continues to accelerate. Cloud spending is expected to exceed over $591B in 2023. How did we arrive at this juncture? How should CIO, CTOs, and engineering teams think about and solve the challenge around cloud spending?

This first article in a two-part series explores the dynamics that have led to the problem of overspending in the cloud, and the next article will explore specific actions Engineering can take to help solve it.

How did we get here?

The stark realities of operating a technology organization in the public cloud have challenged an entire industry. In the legacy world, procurement staff and technology executives were responsible for provisioning private data centers, and the company would reliably be able to say that their data center footprint had a fixed capital cost of servers and equipment purchased, and a fixed monthly cost of hosting space and network bandwidth. All of the company’s applications would be forced into this footprint, and scaling that footprint up or down would be a purchasing process that took place over quarters or even years, with minimal involvement from individual engineering teams and contributors.

Compare this to the public cloud. Individual application teams are now able to make decisions in the public cloud that directly impact cost, and the effects of these decisions can appear on the company’s balance sheet in the next quarter. It is normal to see core applications for a technology company each cost well over $100,000 a month, rolling up to an overall organizational spend that often ranges from $5M to $10M every single year. Decision-making has now shifted to individual teams and contributors from the high-level executives who managed it before. This new responsibility has not generally been framed well by individuals who have inherited it and has led to perceived strain and conflict between engineering and finance teams. Let’s use an allegory to illustrate this.

Imagine you are a successful artisan at a metalworking shop. It is your job to weld, bend, and forge metal to create beautiful and functional art pieces. You have been doing this for 15 years; it is your career. At a certain point, the shop moves to a different location. It is exciting because the new location offers the use of brand-new tools and ovens, which help you do your work better. You eagerly adopt them and enjoy learning about the new state of the art.

Six months later, you are called into the office for a meeting. You are informed that all of the equipment you use to perform your craft has been rented by the hour since the move and that you have been responsible for all the fuel and electricity expended by your welding torches and forging ovens. Your expenditures amount to far too much money and thus must be reduced as soon as possible before the next six-month review!

You leave the meeting feeling resentment, toward the shop’s finance staff, and anxiety around performing your normal job tasks in the future because of the costs that you have been now made hyperaware of. You realize that taking time out of what you consider your “job” to optimize your use of tools and fuel might be required to preserve your gainful employment; however, you see this as an obligation from the higher-ups, rather than an artisanal challenge. Furthermore, the anxiety has no end date, because the workshop will not move again anytime soon, and this will be a constant thorn in your side as you doggedly continue trying to do “your job.”

This is exactly how engineers have felt, suddenly becoming responsible for the cost in the public cloud!

Shifting the culture in engineering

Cost optimization can become an enjoyable problem for developers if you reframe operational costs as an engineering optimization problem. Engineers are more likely to tackle it in the same manner as improving their code’s performance using better algorithms or simpler, more succinct code to accomplish the same tasks by giving the engineering team a tool-based feedback loop to see how their specific costs change as they make tweaks.

FinOps: Finance and Engineering working together

The handoff of spending responsibility to an engineering team must involve trust, support, and communication. The finance team should not be a punitive presence that appears every quarter or twice a year to suddenly attribute hundreds of thousands of dollars of financial responsibility to your team and demand fixes. Instead, there should be an understanding that cost optimization is a journey that both Finance and Engineering are embarking on at the same time, and a tight feedback loop should be created between the roles so that, as the application grows, both teams can understand the technical decisions being made and how that can impact the cost.

The result should be that engineers should still feel free to innovate, experiment, and even justify future expenses, so long as that, in turn, creates a commensurate business value for the company. In turn, Finance can notify Engineering in a timely fashion whenever something seems amiss before things get too out of hand, and work on the problem together in a blameless fashion.

Join us in part 2 of this series as we explore four effective habits for FinOps and engineering teams to motivate them to take action.