Self-managed DBaaS running on Kubernetes with Ondat and SUSE Rancher

The previous blog in this series outlined the challenges of relying on a public cloud database as a service (DBaaS). This blog will outline a compelling alternative: creating a self-managed DBaaS running on Kubernetes with Ondat and SUSE Rancher. 

Running your own DBaaS can reduce the operating cost of running stateful applications on Kubernetes by an order of magnitude. However, current received wisdom holds that running your own DBaaS is eccentric at best - madness at worst! This blog sets out to counter this “wisdom”, challenging the common objections and sealing our arguments with a practical demo. 

The combination of RKE (Rancher Kubernetes Engine) and Ondat, makes it simple and straightforward to create an internal DBaaS solution that is more performant, secure, and reliable than any of the current cloud offerings.

Is my data reliable and safe?.

The most common objection to running any internal DBaaS is data reliability and safety. After all, the cloud is infallible! However, the nasty little secret is that cloud storage is far from foolproof. Let’s start with the numbers. Consider Amazon AWS Elastic Block Storage (EBS). Unless you specifically provision high durability volumes, EBS advertises an annualised failure rate (AFR) of 0.1 % and 0.2 %. By comparison, a representative SAN hard drive has an AFR of 0.3 %. That's not a massive difference in itself and, when drives are combined in a SAN array, the chances of losing data due to hardware failure become imperceptibly small. 

Ondat’s intelligent data mesh continuously replicates data across multiple hardware nodes in real-time. This replication offers two significant advantages. Firstly, it means you don't need dedicated storage hardware to make use of dynamically provisioned Kubernetes volumes. Ondat will ensure that the data is already present when a workload is reprovisioned to a new node. This is not the case with hosted DBaaS, or indeed any networked storage solution from your cloud provider.Here, when Kubernetes shuffles workloads for either efficiency or failover, reattaching storage can require manual interventions and, even without this, can lead to application outages that are longer than expected.

The second major advantage of Ondat’s Kube-Native data architecture is that you can use the screamingly fast local storage on the servers, meaning that latency and performance-sensitive workloads, such as RDBMS, are not bound by network I/O. This extends to the cloud, allowing users to leverage dedicated storage such as AWS i series or Azure LSV series instances to ensure that workloads have access to highly performant storage.

Let's tackle another major piece of received wisdom: creating a DBaaS is insecure when compared to the built-in security of the cloud. This wisdom is, at best, a half-truth. There are indeed powerful security options available in cloud-based DBaaS. For example, both RDS and Azure databases have powerful encryption at rest options. However, and this is important, they are not always enabled by default. You have to be aware of the possibilities, and even if using automation, ensure they are turned on at all times. In effect, every time a new DB instance is created, someone has to be responsible for securing the data.

Running a self managed DBaaS using RKE and Ondat can ensure that your data is fully encrypted at rest by default. Ondat allows administrators to automate data encryption, guaranteeing that it is always enabled for any volumes created within RKE. This security can be extended to include a per-volume encryption key, perfect for multi-tenant RKE clusters. The robust security built into RKE, combined with Ondat, allows administrators to create a DBaaS that ensures security is consistently enabled by default. 

Pay less, get more: performance, reliability and security

Using SUSE Rancher and Ondat gives administrators the power to create secure, reliable, and performant storage for a DBaaS platform. But how do costs stack up against Cloud DBaaS? Exceedingly well as it happens.

The previous blog outlined the expense of operating a cloud DBaaS. Typically, one of two things happens; DBaaS costs are left to mount up, seemingly with no end, or strict controls are imposed, which entirely nullify the advantage of using cloud computing in the first place. However, creating your own DBaaS using RKE and Ondat can allow you to maximise return on investment while still allowing developers, DevOps, and platform engineers the simple self-service and automation they crave. 

By enabling transparent data replication and encryption by default RKE and Ondat allow administrators to choose the best storage medium for the task at hand, rather than having to rely on features in a particular SAN or cloud storage medium. As outlined above, this includes the dedicated storage instances in a cloud provider or blisteringly fast disks on physical hardware. Regardless of medium, operators get the same operational guarantees around data security and resiliency. 

From a cost point of view, this unlocks dramatic savings. Cloud DBaaS is based around the same block storage used by compute instances. Turn the dial to 11 on the performance options for these block storage devices, and the cost rockets skywards. By contrast, dedicated storage instances offer cheaper, more performant storage than any block store equivalent. RKE and Ondat ensure that workloads and the data that support them are reliably replicated among several nodes. It allows for a cheaper, more performant, and secure platform for a DBaaS. Performance, price, resilience, security- you no longer need to choose two, you get them all at once.

Operators for Kube-Native Simplicity

And so, we finally come to the last objection; that it is vastly complex to develop and operate your own DBaaS. 

In the past, this assertion was true. Developing your own DBaaS using open source components involves a massive amount of industrialization, and even automating one database engine into a DBaaS is a giant undertaking. However, Kubernetes operators have taken the status-quo and turned it on its head. Operators allow any Kubernetes administrator to have DBaaS capabilities within minutes. And because operators expose their functionality as a Kubernetes API object, anyone with Kubernetes experience can take advantage of these newfound capabilities using a standard configuration language. For developers, this makes creating a new database instance as simple as including an additional YAML document in their existing Helm or Kustomize deployment. In many cases, this is vastly simpler than automating a cloud DBaaS instance, making it quicker and easier to provision the database. 

There are high-quality operators, both commercial and open source, available for a swathe popular databases, from PostgreSQL to MySQL and even NoSQL databases such as Mongo and Elastic. Each of these allows a Kubernetes administrator to install the Operator and have end-users such as developers, provisioning their own self-service instances within minutes. And the roster of operators is growing fast, and the operator pattern is rapidly entrenching itself as the default when it comes to managing complex software.

For the final blog in this series, we will present a how-to guide on deploying the PostgreSQL operator onto Suse Rancher, using OnDat to ensure that data is reliable, secure, and performant. At the end of this, you will have the beginnings of your own DBaaS, which is cloud-agnostic, easy to use, and offers incredible cost savings in comparison to any cloud equivalent.