Traefik on Kubernetes wrong Client IP on incoming connections

Question

I am running Traefik on Kubernetes and I have create an Ingress with the following configuration:

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: whitelist-ingress
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefix
    traefik.ingress.kubernetes.io/whitelist-source-range: "10.10.10.10/32, 10.10.2.10/23"
    ingress.kubernetes.io/whitelist-x-forwarded-for: "true"
    traefik.ingress.kubernetes.io/preserve-host: "true"
spec:
  rules:
  - host:
    http:
      paths:
      - path: /endpoint
        backend:
          serviceName: endpoint-service
          servicePort: endpoint-port
---

When I do a POST on the above endpoint, Traefik logs that the incoming IP is 172.16.0.1 and so my whitelist is not triggered. Doing an ifconfig I see that IP belongs to Docker

docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.26.0.1  netmask 255.255.0.0  broadcast 172.26.255.255

How can I keep the original IP instead of the docker one?

EDIT

Traefik is exposed as LoadBalancer and the port is 443 over SSL

This is its yml configuration

---
kind: Service
apiVersion: v1
metadata:
  name: traefik
  annotations: {}
    # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
spec:
  selector:
    k8s-app: traefik-ingress
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    name: http
  - protocol: TCP
    port: 443
    targetPort: 443
    name: https
  type: LoadBalancer
  externalTrafficPolicy: Local
  externalIPs:
  - <machine-ip>
---

kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: default
  labels:
    k8s-app: traefik-ingress
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: traefik-ingress
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress
        name: traefik-ingress
    spec:
      hostNetwork: true
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 35
      volumes:
        - name: proxy-certs
          secret:
            secretName: proxy-certs
        - name: traefik-configmap
          configMap:
            name: traefik-configmap
      containers:
      - image: traefik:1.7.6
        name: traefik-ingress
        imagePullPolicy: IfNotPresent
        resources:
          limits:
            cpu: 200m
            memory: 900Mi
          requests:
            cpu: 25m
            memory: 512Mi
        livenessProbe:
          failureThreshold: 2
          httpGet:
            path: /ping
            port: 80
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 5
        readinessProbe:
          failureThreshold: 2
          httpGet:
            path: /ping
            port: 80
            scheme: HTTP
          periodSeconds: 5
        volumeMounts:
          - mountPath: "/ssl"
            name: "proxy-certs"
          - mountPath: "/config"
            name: "traefik-configmap"
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        - name: dashboard
          containerPort: 8080
        args:
        - --logLevel=DEBUG
        - --configfile=/config/traefik.toml
---

As you can see here is the output of kubectl get svc

traefik                       LoadBalancer   10.100.116.42    <machine-ip>     80:30222/TCP,443:31578/TCP   <days-up>

Note that Traefik is running in a single node kubernetes cluster (master/worker on the same node).

You need to add more info, what kind of kubernetes deployment are you testing? which of the traeffik deployment methods are you using ? — mdaguete, Jan 25 '19 at 15:53

score 2 · Answer 1 · answered Jan 25 '19 at 16:16

2

In LoadBalancer service type doc ssl support on aws you can read the following statement:

HTTP and HTTPS will select layer 7 proxying: the ELB will terminate the connection with the user, parse headers and inject the X-Forwarded-For header with the user’s IP address (pods will only see the IP address of the ELB at the other end of its connection) when forwarding requests.

So if you add the following annotation to you traeffik service:

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https

It should work with the ingress.kubernetes.io/whitelist-x-forwarded-for: "true" annotation present in your ingress config and the forwarded header is added by the aws loadbalancer.

Disclaimer: I have not tested that solution.

Regards.

answered Jan 25 '19 at 16:16

mdaguete

387
2
4

I am not running kubernetes on aws but it is on a bare metal; I managed to install it by following this guide https://docs.oracle.com/cd/E52668_01/E88884/html/kubernetes_install_worker.html – Justin Jan 25 '19 at 16:31
So, you'll need some kind of loadbalancer before traeffik (ie haproxy) to add the `x-forwarded-for` header .As LoadBlancer Services expects this kind of service, as in awk, azure, gce, etc. – mdaguete Jan 25 '19 at 16:56
So I Need to provision a haproxy as a replicaset on kubernetes but what kind of kubernetes service should it be? Load Balancer? – Justin Jan 26 '19 at 09:11
I think thay would be easy to provision the loadbalancer outside kubernetes and loadbalance the traeffik ports on k8s nodes ( configuring traeffik with NodePort). – mdaguete Jan 29 '19 at 08:52

Traefik on Kubernetes wrong Client IP on incoming connections

1 Answers1