1

Context

Say we have d.yaml in which a deployment, whose strategy is RollingUpdate, is defined.

We first create a deployment:

kubectl apply -f d.yaml

After some time, we modify d.yaml and re-apply it to update the deployment.

vi d.yaml
kubectl apply -f d.yaml

This starts rolling out a new replicaset R_new.

Normally, the old (previous) replicaset R_old is killed only after R_new has successfully been rolled out.

Question (tl;dr)

Is it possible to kill R_old without waiting for rolling out R_new to complete?

By "kill", I mean completely stopping a replicaset; it should never restart. (So kubectl delete replicaset didn't help.)

Question (long)

In my specific situation, my containers connect to an external database. This single database is also connected from many containers managed by other teams.

If the maximum number of connections allowed is already reached, new containers associated with R_new fail to start (i.e. CrashLoopBackOff).

If I could forcefully kill R_old, the number of connections would be lowered by N where N is the number of replicas, and thus R_new's containers would successfully start.

FAQ:

Q. Why not temporarily stop using RollingUpdate strategy?

A. Actually I have no permission to edit d.yaml. It is edited by CI/CD.

Q. Why not just make the maximum number of connections larger?

A. I have no permission for the database either...

ynn
  • 3,386
  • 2
  • 19
  • 42

2 Answers2

2

Is it possible to kill R_old without waiting for rolling out R_new to complete?By "kill", I mean completely stopping a replicaset; it should never restart. (So kubectl delete replicaset didn't help.)

  1. Make Changes to deployment
  2. Scale down deployment to replicas=0 so that it is as good as stopping the old replicaset
  3. scale up deployment to desired number of replicas, new replicasets will be created with new configuration changes in deployment.

steps number 1 & 2 can be interchanged based on the requirement

confused genius
  • 2,876
  • 2
  • 16
  • 29
1

Technically, you can delete the old replicaset by running kubectl delete replicaset R_old and this would terminate the old pod. I just verified it in my cluster (kubernetes version 1.21.8).

However, terminating a pod doesn't necessarily mean it is been killed immediately. The actual termination of the pod depends on whether or not a preStop lifecycle hook is defined, and on the value of terminationGracePeriodSeconds (see description by typing kubectl explain pod.spec.terminationGracePeriodSeconds).

By default, the kubelet deletes the pod only after all of the processes in its containers are stopped, or the grace period has been over.

Meir Pechthalt
  • 155
  • 4
  • In my case, `kubectl delete replicaset` didn't help. It actually once deleted the specified replicaset, but it was automatically recreated and started right after the deletion. (This is because a replicaset is manually killed but the deployment which manages the replicaset is alive, maybe?) – ynn Mar 26 '22 at 23:29
  • UPDATE: When I try your solution with a minimal setup, it worked as you say. So, the reason why it didn't help in my specific case is maybe because of another configuration which I failed to refer to in the OP... I'll try to find it. – ynn Mar 27 '22 at 14:00