4

I work with a number of Google Cloud Services. Some of the Cloud Storage buckets where I work are in project-A, which is accessible from my own account, and some are in project-B, which I can only access with a service account. I activated the appropriate service account with

gcloud auth activate-service-account --key-file mykey.json

During subsequent logins I just used

gcloud config set account service_account

I also needed to switch my active project, for which I used the command

gcloud config set project project-B

The service account worked perfectly well, allowing me to access a particular bitbucket stored on Google Cloud Storage, but afterward I needed to use Argo to submit a kubernetes workflow, so I switched back to my account and project with the commands 

gcloud config set account my_account
gcloud config set project project-A

Using "gcloud auth list" I can see that my_account is active

ACTIVE  ACCOUNT
        service_account.iam.gserviceaccount.com
*       my_account

But I got the following error when I tried submitting running "kubectl -n default get services":

Error from server (Forbidden): services is forbidden: User "service_account.iam.gserviceaccount.com" cannot list resource "services" in API group "" in the namespace "default": Required "container.services.list" permission.

I get the same error when I use my_namespace rather than default. I'm rather confused by this because gcloud commands like "gsutil ls" function as though I'm in the correct account and project. Why does kubernetes still think I'm using a service account, and how do I make it stop using it so I can do what I need to with kubernetes again? The commands and pipelines I'm trying to run worked perfectly fine before activating the service account.

Nathan Ricke
  • 101

1 Answers1

6

It turns out I just needed to run

gcloud container clusters get-credentials mycluster

Initially, I thought that I would also need to fix this by deleting my kubeconfig file at ~/.kube/config then running the get-credentials command, but as pointed out in the comments it turns out that the get-credentials command was all that was necessary.

The problem appears to specifically surface from running argo commands while the gcloud service account is active: the command would first fail, then all subsequent kubectl commands would return this permissions error. If I switched into the service account, ran the gcloud commands, then switched back into my account, it would not affect kubectl's behavior.

Nathan Ricke
  • 101