Go back to Blog articles
When hearing the product name Redis, those who haven’t looked closely at the technology since its inception may be surprised to learn that Redis has evolved significantly and is much more than an alternative to Memcached as an in-memory key-value datastore.
The Redis homepage provides a great overview of the Redis’ evolution, detailing the platforms versatility as an ‘in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster’
The vast functionality available as standard in Redis lends itself well to a multitude of use-cases and its increased popularity directly relates to its use as an alternative for traditional database RDBMS systems, therefore allowing it to function as an infrastructure component in use-cases including E-Commerce, Financial Services, Internet of Things and Real-Time Analytics.
According to DataDogHQ, Redis is the 2nd most common technology run within Docker (https://www.datadoghq.com/docker-adoption/) –
Data Persistence
Although Redis makes extensive use of memory for both high performance and caching, Data Persistence is also enabled in Redis by default. Redis achieves this through the use of periodic snapshots where the in-memory dataset is asynchronously transferred from memory to disk as a binary dump using the ‘Redis RDB Dump File Format’.
Journaling is also available to Redis and can be used both independently or in conjunction to provide an append-only file, thus further improving data availability and persistence.
Containers / Kubernetes and Ephemeral Storage
Prior to the use of Containers and Kubernetes, applications would run on localised hosts with access to localised disk storage in line with the requirement for the lifecycle of the application. The data being both local and physically persistent.
Containerised applications are often transient with platforms like Kubernetes providing advanced scheduling and recovery. Kubernetes schedules workload to an optimum node with respect to resource utilisation and availability and should a resource fail, recovery is automatic with the application and associated services moved to what may be physically different resources within a Kubernetes cluster.
Without due consideration, data storage in a container orchestrated environment is often ephemeral and although platforms such as Redis go to great lengths for high availability and recovery of data storage, the benefits and advancements are only fully available when used in conjunction with end-to-end advancements in Kubernetes that encompass persistent storage.
Kubernetes CSI and Ondat
Kubernetes provides CSI, the Container Storage Interface as a bridge for the much-needed requirements of persistent storage. An interface that encompasses and allows end-to-end declaration of an application including persistent storage, therefore allowing storage to persist and be available when used in an environment where an application may transition between physical nodes in a highly available cluster.
Ondat, a market leader in the field of cloud native persistent storage provides a Kubernetes native CSI interface, facilitating high performance, data availability and advanced storage features that historically were only available in hardware based enterprise class storage arrays.
The following example demonstrates the use of Redis with Kubernetes, alongside highly available persistent storage from Ondat –
1. Setup and Configuration
For this setup, we have a standard Kubernetes cluster that has StorageOS installed through the one-shot installation process –
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
storageos-k8s1 Ready controlplane,etcd,worker 6d22h v1.17.17
storageos-k8s2 Ready controlplane,etcd,worker 6d22h v1.17.17
storageos-k8s3 Ready controlplane,etcd,worker 6d22h v1.17.17
To provide control over node workload scheduling, we’re selectively using taints and tolerations, forcing our container to run on specific nodes for example purposes. When a node is tainted, a container must be able to tolerate the taint for the container to be scheduled to that node. In our case, none of our containers will have the toleration and only one node, will not have the taint, therefore forcing the Kubernetes scheduler to schedule our container to the untainted node (storageos-k8s1) –
✅ UnTainted storageos-k8s1 - kubectl taint nodes storageos-k8s1 exclusive=true:NoSchedule- --overwrite ⚠️ Tainted storageos-k8s2 - kubectl taint nodes storageos-k8s2 exclusive=true:NoSchedule --overwrite ⚠️ Tainted storageos-k8s3 - kubectl taint nodes storageos-k8s3 exclusive=true:NoSchedule --overwrite
2. Configuration of Redis with Kubernetes
Redis is straightforward to deploy within Kubernetes owing to the ease and availability of official images through Docker Hub. These images translate well to a Kubernetes manifest and in this example, Redis is being deployed as a StatefulSet, a Kubernetes that object that is desirable for Stateful applications. This example makes use of the reference StatefulSet definition from the official Kubernetes documentation with the Redis container image requirements, added to the manifest. The modifications are highlighted for transparency –
✨ Deploying Redis on Kubernetes - kubectl apply -f- <<EOF apiVersion: apps/v1 kind: StatefulSet metadata: name: redis spec: selector: matchLabels: app: redis serviceName: redis replicas: 1 template: metadata: labels: app: redis spec: containers: - name: redis image: redis:6 imagePullPolicy: Always ports: - containerPort: 6379 name: redis EOF statefulset.apps/redis created
3. Verify that Redis is Running
With this simple configuration, we have a Redis deployment running in Kubernetes, albeit without the benefits of persistent storage –
🔍 Checking StatefulSets - kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES redis 1/1 3m49s redis redis:6 🔍 Checking Pods - kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-0 1/1 Running 0 3m49s 10.42.0.123 storageos-k8s1 <none> <none>
4. Redis Default Persistence
By default, Redis provides Persistence through RDB snapshots, we can verify the configuration parameters through a redis-cli query using the running container –
🔎 Checking Redis Persistence kubectl exec -it redis-0 -- sh -c "echo 'config get save' | redis-cli" 1) "save" 2) "3600 1 300 100 60 10000" kubectl exec -it redis-0 -- sh -c "echo 'config get appendonly' | redis-cli" 1) "appendonly" 2) "no"
This output confirms that RDB is enabled. The existing configuration states that should ‘1 change occur in 3600 seconds, a snapshot is taken’, should ‘100 changes occur in 300 seconds, a snapshot is taken’ lastly, should ‘10000 changes occur in 60 seconds’ a snapshot is taken.
5. Enabling the Redis appendonly log
At present the appendonly log is turned off so for enhanced persistence and availability, we tweak the running configuration with the following change as container runtime arguments –
✨ Reconfiguring Redis with RDB and AOF - kubectl apply -f- <<EOF apiVersion: apps/v1 kind: StatefulSet metadata: name: redis spec: selector: matchLabels: app: redis serviceName: redis replicas: 1 template: metadata: labels: app: redis spec: containers: - name: redis image: redis:6 imagePullPolicy: Always args: ["--appendonly", "yes", "--save", "30", "100"] ports: - containerPort: 6379 name: redis EOF statefulset.apps/redis configured
With these changes applied, the container is automatically recreated to match the newly desired configuration –
🔍 Checking StatefulSets - kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES redis 1/1 19m redis redis:6 🔍 Checking Pods - kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-0 1/1 Running 0 45s 10.42.0.124 storageos-k8s1 <none> <none>
n.b. the running time of the StatefulSet which hasn’t changed (19m), and the new running time for the recreated container (45s).
If we were again to check the Redis configuration to confirm persistence parameters, we can verify that our deployment changes are active –
🔎 Checking Redis Persistence kubectl exec -it redis-0 -- sh -c "echo 'config get save' | redis-cli" 1) "save" 2) "30 100" kubectl exec -it redis-0 -- sh -c "echo 'config get appendonly' | redis-cli" 1) "appendonly" 2) "yes"
6. Writing data to Redis and Verifying Persistence
The Redis-Developer community provides a variety of projects and example use-cases for Redis. Within the trove of samples are convenient datasets for populating Redis. Using these we can conveniently populate Redis and check data persistence –
💾 Copying Dataset to redis-0:/tmp - kubectl cp redis-datasets redis-0:/tmp ✅ Importing Redis Datasets - - 🎥 Films - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_movies.redis | redis-cli --pipe" - 🤨 Actors - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_actors.redis | redis-cli --pipe" - 👤 Users - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/user-database/import_users.redis | redis-cli --pipe"
When querying Redis, we can now see that the keyspace is populated with 7605 keys –
🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli" # Keyspace db0:keys=7605,expires=0,avg_ttl=0
It is also possible to see the attempt made by Redis for data persistence by viewing /data to see both the RDB snapshots and the appendonly log configuration from a filesystem viewpoint –
💾 Checking /data - kubectl exec -it redis-0 -- sh -c "ls -alh /data" total 3.9M drwxr-xr-x 2 redis redis 4.0K Mar 30 13:55 . drwxr-xr-x 45 root root 4.0K Mar 30 13:54 .. -rw-r--r-- 1 redis redis 2.3M Mar 30 13:55 appendonly.aof -rw-r--r-- 1 redis redis 1.6M Mar 30 13:55 dump.rdb
This displays a snapshot of the data (dump.rdb) and the appendonly log for transient data. These two files, provide the appropriate means for Redis to recover the in-memory datastore should Redis be restarted.
7. Data loss with Ephemeral Storage
In its current carnation, the loss of the Redis container will subsequently result in the loss of the current Redis keyspace owing to our current implementation within Kubernetes making use of an ephemeral filesystem. For example, deleting the container –
❌ Removing redis pod - kubectl delete pod redis-0 --grace-period=0 pod "redis-0" deleted
Causes the recreation of our container within the StatefulSet
🔍 Checking StatefulSets - kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES redis 1/1 153m redis redis:6 🔍 Checking Pods - kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-0 1/1 Running 0 43s 10.42.0.125 storageos-k8s1 <none> <none>
From a Redis viewpoint, we have experienced dataloss and this is visible by querying the keyspace –
🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli" # Keyspace
8. Using Redis with StorageOS for Persistent Storage
With subtle changes to our deployment, the addition of a StorageOS persistent volume and the reconfiguration of our application to use persistent storage (highlighted in the examples below). Redis can function as expected with highly available persistent storage in the manner in which it was designed –
✨ Recreating Redis with StorageOS Persistent Volumes - 2 Replicas - kubectl apply -f- <<EOF apiVersion: v1 kind: PersistentVolumeClaim metadata: name: task-pv-claim labels: storageos.com/replicas: "2" # 3 Copies of the data - Primary, Replica-1, Replica-2 spec: storageClassName: fast accessModes: - ReadWriteOnce resources: requests: storage: 1Gi --- apiVersion: apps/v1 kind: StatefulSet metadata: name: redis spec: selector: matchLabels: app: redis serviceName: redis replicas: 1 template: metadata: labels: app: redis spec: volumes: - name: task-pv-storage persistentVolumeClaim: claimName: task-pv-claim containers: - name: redis image: redis:6 imagePullPolicy: Always args: ["--appendonly", "yes", "--save", "30", "100"] ports: - containerPort: 6379 name: redis volumeMounts: - mountPath: "/data" name: task-pv-storage EOF persistentvolumeclaim/task-pv-claim created statefulset.apps/redis configured
The storage also has the added benefit of the advanced functionality available in Ondat. In the example it’s demonstrates data replication, but this can be taken further with Ondat features including compression and encryption. We now have a running pod alongside a persistent volume and an active persistent volume claim –
🔍 Checking StatefulSets - kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES redis 1/1 5m4s redis redis:6 🔍 Checking Pods - kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-0 1/1 Running 0 74s 10.42.2.87 storageos-k8s2 <none> <none> 🔍 Checking PV's - kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-1a0c108f-5447-409b-a7c7-e18c682710eb 1Gi RWO Delete Bound default/task-pv-claim fast 5m4s 🔍 Checking PVC's - kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE task-pv-claim Bound pvc-1a0c108f-5447-409b-a7c7-e18c682710eb 1Gi RWO fast 5m5s
Redis is again repopulated, using the example datasets. This time, persistent actions within Redis to the /data filesystem are saved in Persistent Storage –
💾 Copying Dataset to redis-0:/tmp - kubectl cp redis-datasets redis-0:/tmp ✅ Importing Redis Datasets - - 🎥 Films - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_movies.redis | redis-cli --pipe" - 🤨 Actors - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/movie-database/import_actors.redis | redis-cli --pipe" - 👤 Users - kubectl exec -it redis-0 -- sh -c "cat /tmp/redis-datasets/user-database/import_users.redis | redis-cli --pipe"
9. Demonstrating Data Persistence and High Availability with Node Failure
Currently, the container through taints and tolerations is forced to schedule on storageos-k8s1. The taints and tolerations are adjusted so that any future scheduling will target storageos-k8s2, an independent node with independent cpu/memory resources and storage
⚠️ Tainted storageos-k8s1 - kubectl taint nodes storageos-k8s2 exclusive=true:NoSchedule --overwrite ✅ UnTainted storageos-k8s2 - kubectl taint nodes storageos-k8s1 exclusive=true:NoSchedule- --overwrite ⚠️ Tainted storageos-k8s3 - kubectl taint nodes storageos-k8s3 exclusive=true:NoSchedule --overwrite
Deleting the pod then causes the scheduler to recreate the container on storageos-k8s-2
❌ Removing redis pod - kubectl delete pod redis-0 --grace-period=0 pod "redis-0" deleted
Although the container is recreated and is now running on storageos-k8s2, the persistent volume and persistent volume claim remain intact.
🔍 Checking StatefulSets - kubectl get statefulsets -o wide NAME READY AGE CONTAINERS IMAGES redis 1/1 5m4s redis redis:6 🔍 Checking Pods - kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES redis-0 1/1 Running 0 74s 10.42.2.87 storageos-k8s2 <none> <none> 🔍 Checking PV's - kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-1a0c108f-5447-409b-a7c7-e18c682710eb 1Gi RWO Delete Bound default/task-pv-claim fast 5m4s 🔍 Checking PVC's - kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE task-pv-claim Bound pvc-1a0c108f-5447-409b-a7c7-e18c682710eb 1Gi RWO fast 5m5s
Finally, we can verify that the data is available by checking the available keys, on the newly created container –
🔑 Checking Redis Keys - kubectl exec -it redis-0 -- sh -c "echo 'INFO KEYSPACE' | redis-cli" # Keyspace db0:keys=7605,expires=0,avg_ttl=0
The result, 7605 keys restored and available through Redis and StorageOS persistent storage.
We hope that you have found this demonstration and viewpoint of both Redis and Ondat useful. More information on Redis can be found at redis.io and should you wish to try Ondat, you can test drive it with the Free Developer Edition providing 5TB for evaluation purposes.