It’s a tech cliché that every CEO starts a new year with predictions that the coming year is: “The year of [whatever my company focuses on].” I will spare you the “2022…stateful Kubernetes applications” spiel for a couple of reasons: Firstly, users have been building stateful applications in Kube for a while now. And secondly, because being overly self-absorbed is not a healthy trait for any company - it pays to have a view of your surroundings.
The Cost of Getting State Wrong
Running stateful, often business-critical applications in Kubernetes is now increasingly common. But there is a big difference between doing something and doing it well. Early-stage architectural decisions on how to implement stateful applications in Kube are pretty technical, but getting them wrong can cost dearly - in very clear business terms:
- Lock-in, whether it is to cloud providers, hardware solutions, or managed data and storage services, translates pretty cleanly into inefficient, uncompetitive operating costs. Poor storage choices can inflate Cloud costs by several orders of magnitude, and the amount of data buinesses have to handle isn’t going down.
- Ruthless competition in digital business means that poor, laggy application performance and an inability to scale rapidly lead to increased customer churn.
- System downtime, service outages, and poor security?... Let’s not point fingers, but there are a growing number of CEOs you can ask about the total cost to the business from losing customer confidence or, worse still, customer data.
My prediction for the year is that, increasingly, executives overseeing digitization, product owners, system architects, platform engineers, and cloud developers will have to start caring more about how state is done in Kube. “Doing” state is about how we store state and, in many Cloud Native environments, this has been going ahead away from the watch of the traditional enterprise storage engineers and experts.
CNCF Storage TAG
Let’s segway briefly to my second point: having a good view of your business surroundings. In my role as chair of the CNCF Storage Technical Advisory Group (TAG), I am privileged to have an incredible view of what is happening around state and storage within the context of Kubernetes and Cloud-Native computing. All technical decisions in the CNCF are taken by the elected Technical Operating Committee (TOC). However, as the foundation has grown, the TOC has needed more specialist support to scale. This is not to make decisions but to deliver advice, provide subject matter knowledge and expertise, help the TOC review projects, and provide content for the community.
A Useful Resource
This brings us back to my prediction that more stakeholders in the Kube Native world need to care about state and storage. Our most significant milestone to date in the CNCF Storage TAG was when we created the Storage Landscape Whitepaper. This was intended to provide an overview of what can often seem like an impenetrable world of overlapping terms, metrics, and considerations.
Its objective was to provide greater clarity for the increasing number of non-storage-experts who, in a Cloud/Kube-Native world of DevOps automation and Infrastructure-as-Code (IaC), were getting more and more involved, making important decisions, yet typically without having the understanding of some of these environments.
The paper covers basic storage attributes, such as availability, scale, performance, data protection, and failovers - the kind of terms every IT salesperson will hit you with. But what we tried to overcome was how these terms can be highly generic and multifaceted when you scratch below the surface. We then provided an overview of the basic technology landscape, storage interfaces, and the options for delivering state into Kube.
Even with the incredible team I worked with on the Storage TAG, it took several attempts to create a meaningful taxonomy for these basic aspects. But with this in the bag, we developed some valuable tabulations for exploring the attributes of different storage options more meaningfully and in greater context.
I am not suggesting that busy digitization team leads and executives need to digest all 44 pages of the paper (although it should be intelligible for a non-engineer with a little tech-savvy). It is just that, in a Dev ‘X’ Ops world of intermeshed, overlapping roles and responsibilities, it is vital that architects, platform engineers, and key decision-makers understand some of the implications of storage decisions in more detail (even recommend just skim-reading the use case/comparison tables and using the text to drill down into anything they don’t understand would be useful).
At the other end of the scale, effective DevOps automation should mean that developers don’t have to concern themselves with storage or any other aspect of infrastructure and operations. Kubernetes just want fast, simple access to persistent volumes, their favourite databases, and stateful apps. And with the right solutions for storage management, optimization, automation, and developer self-service, this is what they can have. But someone on the team needs to understand this need and know what the “right” solution is and why it’s right.