Kubernetes ingress-nginx gives 502 error (Bad Gateway)

Question

I have an EKS cluster for which I want : - 1 Load Balancer per cluster, - Ingress rules to direct to the right namespace and the right service.

I have been following this guide : https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes

My deployments:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
      - name: hello-world
        image: IMAGENAME
        ports:
        - containerPort: 8000
          name: hello-world


---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bleble
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: bleble
  template:
    metadata:
      labels:
        app: bleble
    spec:
      containers:
      - name: bleble
        image: IMAGENAME
        ports:
        - containerPort: 8000
          name: bleble

the service of those deployments:

apiVersion: v1
kind: Service
metadata: 
  name: hello-world-svc
spec: 
  ports: 
     -  port: 8080
        protocol: TCP
        targetPort: 8000
  selector: 
    app: hello-world
  type: NodePort

---

apiVersion: v1
kind: Service
metadata: 
  name: bleble-svc
spec: 
  ports: 
     -  port: 8080
        protocol: TCP
        targetPort: 8000
  selector: 
    app: bleble
  type: NodePort

My Load balancer:

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http

My ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: simple-fanout-example
  namespace : default
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: internal-lb.aws.com
    http:
      paths:
      - path: /bleble
        backend:
          serviceName: bleble-svc
          servicePort: 80
      - path: /hello-world
        backend:
          serviceName: hello-world-svc
          servicePort: 80

I've set up the Nginx Ingress Controller with this : kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.24.1/deploy/mandatory.yaml

I am unsure why I get a 503 Service Temporarily Unavailable for one service and one 502 for another... I would guess it's a problem of ports or of namespace? In the guide, they don't define namespace for the deployment...

Every resources create correctly, and I think the ingress is actually working but is getting confused where to go.

Thanks for your help!

Answer 1

In general, use externalTrafficPolicy: Cluster instead of Local. You can gain some performance (latency) improvement by using Local but you need to configure those pod allocations with a lot efforts. You will hit 5xx errors with those misconfigurations. In addition, Cluster is the default option for externalTrafficPolicy.

In your ingress, you route /bleble to service bleble, but your service name is actually bleble-svc. please make them consistent. Also, you would need to set your servicePort to 8080 as you exposed 8080 in your service configuration.

For internal service like bleble-svc, Cluster IP is good enough in your case as it does not need external access.

Hope this helps.