This series of posts will walk you through a hands-on experience focused on building a simple (but cool) stateful application that has been designed to run on Kubernetes. Part 1 sets the stage.
Provisioning stateful applications on-demand in Kubernetes can be challenging. Stateful apps are built on databases, message buses, or similar systems that persist data to disk. If you think, “Well, it’s almost the case for every application,” then you’re right, but the gotcha is that generally, the backend doesn’t live in Kubernetes.
In addition, their architecture is generally spread across heterogeneous platforms. Typically, the frontend component already sits in Kubernetes and can be provisioned, scaled up, and down on-demand as it is usually stateless. At the same time, data sets can be hosted in various formats and form-factors: Virtual Machine on-premises, PaaS services in the Public Cloud, Big Data platform, or even physical machines.
This series explores the DevOps experience whendeveloping stateful applications designed to run on Kubernetes and building on-demand data services.Why run all components in Kubernetes, you may ask. The goal is for application owners to accelerate the release of business-critical applications, taking advantage of native Kubernetes benefits.
We’re going to focus on the toolset that enables developers to navigate through the development lifecycle of these applications efficiently. Namely, we’re going to delve into:
Kustomize
Skaffold
Ondat Persistent Volumes
MongoDB Database and Replica Set
MongoDB Community Kubernetes Operator
Pymongo
If this is not the most DevOps setup for a stateful app, I don’t know what DevOps means anymore 😆.
Meet the Marvel App
We are going to build an app that displays Marvel characters cards selected randomly to demonstrate Kubernetes capabilities. Our goal is also to explore how DevOps tools fit in the application development process. Hopefully, this will be a lot more fun than yet another “Hello World” application! The app architecture is represented below:
The FE (Frontend) component is a Python Flask application. Its role is to provide a visualization layer for data ingested into the MongoDB database. It runs in Kubernetes as aDeployment.
The BE (Backend) is a MongoDB database deployed as a 3-node cluster and managed by the MongoDB Community Kubernetes Operator. It runs in Kubernetes as of aStatefulSet, which is standard for an application that needs to persist data to disk.
As shown in the picture, aStatefulSetleverages a Kubernetes controller ensuring that eachPodhas access to its own datastore by claiming persistent volumes (PV) tied to individualPVCs. For more information onStatefulSetsand their use cases, you can take a lookhereon the Ondat website or in the official Kubernetesdocumentation.
As part of the application deployment, a KubernetesJobis also provisioned. Its role is to fetch Marvel characters’ information from the Marvel API availablehereand store it in the MongoDB database. TheJobis only run once, at application deployment time. Kubernetes will schedulePodsuntil the job is successfully completed (within a limited number of retries).Podsmay fail, waiting for the connection to the database to be established, but once succeeded, it is never rerun. The FE can then display random characters cards directly retrieved from the database.
The application comprises the FE +BE and a data initialization step that populates the BE with relevant information. It can be deployed in Kubernetes using YAML manifests dynamically generated by a command-line tool or as a CI/CD pipeline. Kustomize is our tool of choice in this article for generating manifests. It can also update objects related to a particular build iteration. With Kustomize, you can easily update container image references, or for our use case, deploy the application with a new version of Mongo, or provision a new OndatStorageClass.
Templatize and Automate All the Things!
It all begins with the ability to iterate application testing and deployment in various development stages, such as locally on a laptop, on a remote testing cluster, or during staging, User Acceptance Testing (UAT), and production phases. Kustomize can be used as akubectl option (-k) to apply customization when generating application manifests. This tool allows developers to dynamically adapt application requirements and context to specific environments. However, we generally recommend using Kustomize as a separate binary since it is more flexible and always up to date.
The principle of Kustomize is to build an overlay by specifying the elements from the base manifests you want to modify. In this article, we focus on the application development phase, where we code and test the application within a local Kubernetes K3s cluster. In that use case, Kustomize needs 2 folders, one for the base manifests and one for theDevoverlay:
The content of these files can be accessedhere. One can simply run Kustomize with thebuild option to generate the appropriate Kubernetes manifests. For example, to modify the base manifests with thedevoverlay, you can runkustomize build overlay/dev, assuming you’re in the parent folder of the “overlay” directory. The output is a set of manifests directly displayed on your terminal, so if you want to save the result as YAML, just redirect the output to a file. Another option is to use the output ofkustomize buildas input for kubectl applylike this:
kustomize build overlay/dev | kubectl apply -f -
It will directly deploy the objects into your Kubernetes cluster.
Aligning the Moving Parts
When developing an application designed to run on Kubernetes, you will face repetitive tasks very early. They include building new container images as you commit new code, updating the Kubernetes manifests, and deploying a new version of your application stack into the Kubernetes test cluster so you can execute your tests.
Our goal is to provide a pipeline that automatically builds the container images, updates the manifests accordingly, andcontinuouslydeploys the application stack as soon as we modify the source code. The workflow looks like this:
Skaffold is one of the tools that allows you to do just that. It is an open-source project by Google that provides a CLI to manage the lifecycle of your application within various stages of the CI/CD pipeline. It can help with your app’s development, build, and deployment phases. In our use case, we’re interested in the early dev stage. The role of Skaffold will be to build a new image from the FE Dockerfile every time the code is saved locally before performing. git commit and deploying it to the dev Kubernetes cluster using Kustomize. As a result, you don’t need to commit or push code to your git repository to test it.
Install the dependencies required by the Flask application
Copy the source code to the container
Run the web server usinggunicorn WSGI (as the app doesn’t server any static HTML pages, there’s no need fornginxor another HTTP server).
The Skaffolddevmode allows you to detect any change happening in real-time in the application source code, automatically build a new container image using that Dockerfile, and deploy it to the dev Kubernetes cluster. There’s no need to perform git commitor git pushto trigger this process via a webhook. In that mode, the Skaffold binary is running as a daemon that detects code changes. Skaffold can deploy the application components using different tools. We have chosen Kustomize, but Docker, kubectl, and Helm are also available options. Similarly, the Skaffold build phase can leverage Dockerfiles,buildpacks, and other tools mentioned in thedocumentation, as well as custom scripts.
Since we’re using an ARM-based architecture for development, we need a custom script to perform Docker cross-platform builds. An example of such a script is givenhere. Thebuild.shscript we use is located at the root of the Marvel apprepository. It contains the same code as in the example. Skaffold utilizes this script to build the image artifact in the build phase. Then, Kustomize dynamically generates the Kubernetes manifests, and Skaffold deploys them to the cluster.
If your machine runs on an x86 processor, you don’t need any custom script or additional build commands. But don’t worry, we’ll dive into every component in detail!
Next time, we’ll get our hands dirty by going through every single step required to build this pipeline. We’ll also dig into the MongoDB Operator and explain why you need it!
written by: Nic Vermandé
Nic is an experienced hands-on technologist, evangelist and product owner who has been working in the fields of Cloud-Native technologies, Open Source Software, Virtualization and Datacenter networking for the past 17 years. Passionate about enabling users and building cool tech solving real-life problems, you’ll often see him speaking at global tech conferences and online events, spreading the word and walking the walk with customers.
Register for our SaaS Platform
Learn how Ondat can help you scale persistent workloads on Kubernetes.