1

I am working on a service (written in Go), which is expected to receive a huge number of requests. As per the architecture, each pod of the service is supposed to serve specific clients. Lets say, if there are 3 pods of this service, the split will be like -> A-H, I-P, Q-Z, where each letter is client's name's first letter.

But if there are 4 pods of this service, then split can be -> A-F, G-N, O-U, V-Z.

Is there a way I can know in Go code how many other replicas are there?

PS: AFAIK, one possibility is to have an environment variable in deployment.yaml. But there are ways where scaling can be done without changing the yaml.

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
Aakash Goyal
  • 1,051
  • 4
  • 12
  • 44
  • You probably want [`StatefulSet`](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) instead – zerkms Oct 10 '19 at 07:50
  • 7
    I would argue that the architecture is flawed. Especially when you expect a huge number of requests, I would refrain from adding additional logic into the load balancing and allow all instances to handle any request, then add a tested autoscaling. Let Kubernetes and your application do their respective jobs (scaling and load balancing for Kubernetes, serving requests for your application) in the setup and do not try to reinvent built in functionality of one with the other. – Markus W Mahlberg Oct 10 '19 at 08:03
  • No, this is not possible. Because the number of pods can, and often will, and should, change--particularly during upgrades, or in case of a crash or other restart. You should write your services to be completely agnostic of this. If your service depends on this information, your service needs to be redesigned. Fundamentally, what you're talking about aren't replicas at all--they are stateful, uniquely configured pods. – Jonathan Hall Oct 10 '19 at 08:45
  • @MarkusWMahlberg: Actually, once a pod starts serving a particular customer, all further requests are supposed to be served by that pod only as we need details of previous request to process the current request. That is why we are thinking of this option. Another option we are thinking is to store the information in DB so that any pod can serve the request, but the problem is that the number of requests to DB will be huge and old data also will be little heavy. – Aakash Goyal Oct 10 '19 at 09:09
  • @Gauranga: What happens when that pod goes away? – Jonathan Hall Oct 10 '19 at 09:26
  • What kind of data are you storing? Can you use a database that scales well? What is the nature of the requests? If they're mostly read requests, can each pod keep a read-only copy of the database, which is synced on startup, or something similar? – Jonathan Hall Oct 10 '19 at 09:27
  • They are mostly update requests. So, I need to know the previous data. Yes, we are planning to check for Redis if that works. But AFAIK, we cant bring up Redis cluster using Helm and it needs manual action to do that. Checking more on that. Still working on restart case. – Aakash Goyal Oct 10 '19 at 09:33
  • Persistent storage (whether Redis, or a RDBMS, etc) typically should be set up _outside of Helm/Kubernetes_. While it's possible to do so within K8s, it's more complex to maintain, and often performance suffers (but if you really know what you're doing, maybe you can get it to work). General wisdom is: Use K8s for stateless pods. – Jonathan Hall Oct 10 '19 at 09:38
  • What evidence do you have that a single database can't handle your load? – Jonathan Hall Oct 10 '19 at 09:39
  • Our customers will keep increasing in future, so maintaining their data in storage will reach its limit someday. Now we have only 1500 customers, and data wont be more than 100MB. So, I was thinking more from future perspective. – Aakash Goyal Oct 10 '19 at 09:42
  • 1
    Again: Your architecture is flawed. If you want to separate customers, this should be done via name spacing. Write a helm chart, and setting up a new customer becomes a breeze. @Flimzy I can only partially agree with stateless pods. It is well possible to have stateful pods, even at large scale. You only need to know your DB well. Running Postgres or - god forbid! - ES on k8s? Not so much. MongoDB, Cockroach, Influx? Not a problem, at all. – Markus W Mahlberg Oct 10 '19 at 10:00
  • 2
    Premature optimization is the root of all evil. Write an app that works for the next 6 months. Then when you need to scale up, worry about that then. – Jonathan Hall Oct 10 '19 at 10:02
  • 1
    @MarkusWMahlberg: I know it's possible (I use them myself). But it's not for k8s newbies. This is why I was careful not to claim a hard and fast rule. :) – Jonathan Hall Oct 10 '19 at 10:02
  • I actually think in some situations, given that the program is extremely simple, the "split my load based on current number of replicas" is a perfectly valid, useful way to solve a problem. i am forced to use zookeeper (or similar solution) to register when one of the pods comes up, and unregister when it goes down. also will timeout if not heard from in a certain amount of time. but this adds some complexity and it's a waste if i could just ask "how many replicas i got right now?" in simple "divvy up my work to multiple replicas" - _GetCurrentReplicas_ ability would be very valuable. – Nicholas DiPiazza Nov 11 '20 at 05:08

1 Answers1

1

As per the title, the solution is to use StatefulSet where each service is aware of each other and apps can be written in a way that they handle this scenario.

However, for this question, as per the details mentioned, one good solution without using StatefulSet is to create a Service with a sessionAffinity: ClientIP. The requirement as per the details is that subsequent requests must go to a specific pod which served the previous request. This can be configured using sessionAffinity field. Check documentation for it here With this, when a new client connects, service will select a pod after doing load-balancing. Post that, all subsequent requests will go to that pod only. This can be configured further using SessionAffinityConfig.

Aakash Goyal
  • 1,051
  • 4
  • 12
  • 44