How to install nginx-ingress with hostNetwork on bare-metal?

nginx kubernetes kubernetes-ingress bare-metal-server
staticdev picture staticdev · Jul 6, 2019 · Viewed 9.3k times · Source

I created a single-node kubeadm cluster on bare-metal and after some research I would go for a host network approach (https://kubernetes.github.io/ingress-nginx/deploy/baremetal/#via-the-host-network), since NodePort is not an option due to network restrictions.

I tried installing nginx-ingress with helm chart through the command:

   helm install stable/nginx-ingress \
     --set controller.hostNetwork=true

The problem is that it is creating a LoadBalancer service which is Pending forever and my ingress objects are not being routed:

NAME                                                                 READY   STATUS    RESTARTS   AGE
pod/whopping-kitten-nginx-ingress-controller-5db858b48c-dp2j8        1/1     Running   0          5m34s
pod/whopping-kitten-nginx-ingress-default-backend-5c574f4449-dr4xm   1/1     Running   0          5m34s

NAME                                                    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/kubernetes                                      ClusterIP      10.96.0.1       <none>        443/TCP                      6m43s
service/whopping-kitten-nginx-ingress-controller        LoadBalancer   10.97.143.40    <pending>     80:30068/TCP,443:30663/TCP   5m34s
service/whopping-kitten-nginx-ingress-default-backend   ClusterIP      10.106.217.96   <none>        80/TCP                       5m34s

NAME                                                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/whopping-kitten-nginx-ingress-controller        1/1     1            1           5m34s
deployment.apps/whopping-kitten-nginx-ingress-default-backend   1/1     1            1           5m34s

NAME                                                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/whopping-kitten-nginx-ingress-controller-5db858b48c        1         1         1       5m34s
replicaset.apps/whopping-kitten-nginx-ingress-default-backend-5c574f4449   1         1         1       5m34s

Is there any other configuration that needs to be done to succeed in this approach?

UPDATE: here are the logs for the ingress-controller pod

-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:    0.24.1
  Build:      git-ce418168f
  Repository: https://github.com/kubernetes/ingress-nginx
-------------------------------------------------------------------------------

I0707 19:02:50.552631       6 flags.go:185] Watching for Ingress class: nginx
W0707 19:02:50.552882       6 flags.go:214] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
nginx version: nginx/1.15.10
W0707 19:02:50.556215       6 client_config.go:549] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0707 19:02:50.556368       6 main.go:205] Creating API client for https://10.96.0.1:443
I0707 19:02:50.562296       6 main.go:249] Running in Kubernetes cluster version v1.15 (v1.15.0) - git (clean) commit e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529 - platform linux/amd64
I0707 19:02:51.357524       6 main.go:102] Validated default/precise-bunny-nginx-ingress-default-backend as the default backend.
I0707 19:02:51.832384       6 main.go:124] Created fake certificate with PemFileName: /etc/ingress-controller/ssl/default-fake-certificate.pem
W0707 19:02:53.516654       6 store.go:613] Unexpected error reading configuration configmap: configmaps "precise-bunny-nginx-ingress-controller" not found
I0707 19:02:53.527297       6 nginx.go:265] Starting NGINX Ingress controller
I0707 19:02:54.630002       6 event.go:209] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"staging-ingress", UID:"9852d27b-d8ad-4410-9fa0-57b92fdd6f90", APIVersion:"extensions/v1beta1", ResourceVersion:"801", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress default/staging-ingress
I0707 19:02:54.727989       6 nginx.go:311] Starting NGINX process
I0707 19:02:54.728249       6 leaderelection.go:217] attempting to acquire leader lease  default/ingress-controller-leader-nginx...
W0707 19:02:54.729235       6 controller.go:373] Service "default/precise-bunny-nginx-ingress-default-backend" does not have any active Endpoint
W0707 19:02:54.729334       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:02:54.729442       6 controller.go:797] Service "default/test" does not have any active Endpoint.
I0707 19:02:54.729535       6 controller.go:170] Configuration changes detected, backend reload required.
I0707 19:02:54.891620       6 controller.go:188] Backend successfully reloaded.
I0707 19:02:54.891654       6 controller.go:202] Initial sync, sleeping for 1 second.
I0707 19:02:54.948639       6 leaderelection.go:227] successfully acquired lease default/ingress-controller-leader-nginx
I0707 19:02:54.949148       6 status.go:86] new leader elected: precise-bunny-nginx-ingress-controller-679b9557ff-n57mc
[07/Jul/2019:19:02:55 +0000]TCP200000.000
W0707 19:02:58.062645       6 controller.go:373] Service "default/precise-bunny-nginx-ingress-default-backend" does not have any active Endpoint
W0707 19:02:58.062676       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:02:58.062686       6 controller.go:797] Service "default/test" does not have any active Endpoint.
W0707 19:03:02.406151       6 controller.go:373] Service "default/precise-bunny-nginx-ingress-default-backend" does not have any active Endpoint
W0707 19:03:02.406188       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:03:02.406357       6 controller.go:797] Service "default/test" does not have any active Endpoint.
[07/Jul/2019:19:03:02 +0000]TCP200000.000
W0707 19:03:05.739438       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:03:05.739467       6 controller.go:797] Service "default/test" does not have any active Endpoint.
[07/Jul/2019:19:03:05 +0000]TCP200000.001
W0707 19:03:09.072793       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:03:09.072820       6 controller.go:797] Service "default/test" does not have any active Endpoint.
W0707 19:03:12.406121       6 controller.go:797] Service "default/face" does not have any active Endpoint.
W0707 19:03:12.406143       6 controller.go:797] Service "default/test" does not have any active Endpoint.
[07/Jul/2019:19:03:15 +0000]TCP200000.000
I0707 19:03:54.959607       6 status.go:295] updating Ingress default/staging-ingress status from [] to [{ }]
I0707 19:03:54.961925       6 event.go:209] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"staging-ingress", UID:"9852d27b-d8ad-4410-9fa0-57b92fdd6f90", APIVersion:"extensions/v1beta1", ResourceVersion:"1033", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress default/staging-ingress

Answer

staticdev picture staticdev · Jul 12, 2019

@ijaz-ahmad-khan @vkr gave good ideas for solving the problem but the complete steps for setup are:

1) Install nginx-ingress with: 

helm install stable/nginx-ingress --set controller.hostNetwork=true,controller.service.type="",controller.kind=DaemonSet

2) In your deployments put:

spec:
  template:
    spec:
      hostNetwork: true

3) In all your Ingress objects put:

metadata:
  annotations:
    kubernetes.io/ingress.class: "nginx"

Related questions

Kubernetes Nginx Ingress not finding service endpoint

I'm having some trouble getting the Nginx ingress controller working in my Kubernetes cluster. I have created the nginx-ingress deployments, services, roles, etc., according to https://kubernetes.github.io/ingress-nginx/deploy/ I also deployed a simple hello-world app which listens …

Read More
Nginx ingress controller websocket support

Recently I've been working on a toy app using Kubernetes. Part of the app is a web server that needs to support WebSockets. Currently, I'm using port-forwarding to access the web server and everything works just fine. I'd like to …

Read More
Kubernetes Ingress non-root path 404 Not Found

I have the following that config that works when I try <NodeIP>:30080 apiVersion: extensions/v1beta1 kind: Deployment metadata: name: app-deployment spec: replicas: 3 template: metadata: labels: name: app-node spec: containers: - name: app image: myregistry.net/repo/app:…

Read More