Kubernetes rolling update without downtime?

Question

According to https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#scaling-a-statefulset, I would like to ask how to achieve zero-downtime rolling update? I guess here are the minimum requirements:

(1) .spec.updateStrategy set to RollingUpdate

(2) .spec.podManagementPolicy set to OrderedReady

(3) .spec.replicas set to 2

Is that right? And I assume that when update is happening in reverse order, all traffic to the StatefulSet is served by the pods with lower ordinal number?

Answer 1

Yes to have have zero downtime for statefulsets upgrade, you should have all the points mentioned:

1. .spec.updateStrategy set to RollingUpdate
2. .spec.podManagementPolicy set to OrderedReady which is by default OrderedReady
3. .spec.replicas set to minimum 2.

Another, thing you need to make sure your statefulset doesn't have downtime is proper readiness probe set. The readiness probe tells kubernetes controller manager that this pod is ready to serve request and you can start sending the requests to it.

The reason it is very important while doing zero downtime upgrade is lets say you have two replica's of statefulset and you started rolling upgrade without readiness probe set. The kubernetes will delete the pod in reverse order and make it come to running state and mark it as ready and terminate another pod. Now lets say your container process didn't come up in that time there will be no pod to serve the requests, because one pod is not completely ready yet and kubernetes has terminated another pod for upgrade process and hence the data loss.

readinessProbe:
  httpGet:
    path: /
    port: 80
  initialDelaySeconds: 5
  periodSeconds: 5
  successThreshold: 1

EDIT: The following json snippet I use for rolling update of statefulsets in my case:

 "spec": {
         "containers": [
           {
             "name": "md",
             "image": "",
             "imagePullPolicy": "IfNotPresent",
             "command": [
               "/bin/sh",
               "-c"
             ],
             "args": [
               "chmod -R 777 /logs/; /on_start.sh"
             ],
             "readinessProbe": {
                "exec": {
                   "command": [
                      "cat",
                      "/tmp/ready.txt"
                   ]
                 },
                 "failureThreshold": 10,
                 "initialDelaySeconds": 5,
                 "periodSeconds": 5,
                 "successThreshold": 1,
                 "timeoutSeconds": 1
             },
             "securityContext": {
               "privileged": true
             }
      }

This is how you can setup readiness probe in your statefulset containers. I am setting readiness probe as linux command, if you have http probe then it will be different.

Answer 2

I agree with @Prafull Ladha, the main role of readinessProbe here to guarantee that the new pods created during RollingUpdate are ready to take on requests before terminating the old pods. However, you can also control the rolling update process by specifying the appropriate optional parameters as described in official Kubernetes documentation.