kubernetes ingress-nginx-controller: Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io" no route to host

Question

After creating a deployment for a web-application, consisting of pod, internal service and ingress, the ingress can't be created. When executing kubectl apply -f web-app.yaml, I get the error: error when creating "web-app.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": dial tcp 10.100.7.97:443: connect: no route to host

After some retries, it worked surprisingly, but after some changes to the deployment, the same error occurs again.

web-app.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: development
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web-app
        image: registry/web-app:latest
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: web-app-internal
  namespace: development
spec:
  selector:
      app: web-app
  ports:
  - port: 8080
    targetPort: 8080
    protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: 
    kubernetes.io/ingress.class: "nginx"
  name: web-app-ingress
  namespace: development
  labels:
    name: web-app-ingress
spec:
  rules:
  - host: web.app
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: web-app-internal
            port: 
              number: 8080

Any clues what the issue could be?

Answer 1

The error you are getting is from the webhook trying to validate your ingress, and not the ingress object itself you are trying to apply.

The validation webhook is set by default to fail the request if the webhook cannot process it. Is there are a chance your ingress controller was unavailable for any reason while you were trying to apply the ingress object?

Also, looking at a related SO question, it might be an error related to DNS resolution.

Answer 2

Download and edit the yaml file : wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.2/deploy/static/provider/baremetal/deploy.yaml

Add v1beta1 to admissionReviewVersions and change from v1 to v1beta1 to apiVersions:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.1.2
    helm.sh/chart: ingress-nginx-4.0.18
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  - v1beta1      #<--------- add v1beta1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1beta1   #<--------- change from v1 to v1beta1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None