Kubernetes kube-dns pod is pending

Lakmal Vithanage picture Lakmal Vithanage · Feb 14, 2017 · Viewed 13.1k times · Source

I tried this doc to install and setup Kubernetes in Ubuntu VM. I have finished upto 3/4 and now kube-dns pod is in pending status. How can i figure out this? here is the result for kubectl get pods --namespace=kube-system and kubectl describe pod <pod name>

# kubectl get pods --namespace=kube-system
NAME                              READY     STATUS    RESTARTS   AGE
dummy-2088944543-jk2t2            1/1       Running   0          3h
etcd-ubuntu                       1/1       Running   0          3h
kube-apiserver-ubuntu             1/1       Running   0          3h
kube-controller-manager-ubuntu    1/1       Running   0          3h
kube-discovery-1769846148-h88v4   1/1       Running   0          3h
kube-dns-2924299975-dfp17         0/4       Pending   0          3h
kube-proxy-zdcxw                  1/1       Running   0          3h
kube-scheduler-ubuntu             1/1       Running   0          3h
weave-net-xwfhj                   2/2       Running   0          2h

# kubectl describe pod kube-dns-2924299975-dfp17
Error from server (NotFound): pods "kube-dns-2924299975-dfp17" not found

Answer

Antoine Cotten picture Antoine Cotten · Feb 14, 2017

Cause

Most likely a lack of available computing resources in your cluster.

If you're using the example in cluster/addons/dns you're certainly using a Deployment with resource requests, highlighted if you click the link. It could be that your other pods are already requesting all the available resources in the cluster, therefore your pod doesn't get scheduled.

You can confirm that theory with kubectl --namespace=kube-system describe pod kube-dns-2924299975-dfp17 and look for the following event:

Reason                Message
------                -------
FailedScheduling      pod (kube-dns-2924299975-dfp17) failed to fit in any node
fit failure summary on nodes : Insufficient cpu (3)

You can also describe your nodes with kubectl describe node <node-name> and look at the last information:

Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.
  CPU Requests  CPU Limits      Memory Requests Memory Limits
  ------------  ----------      --------------- -------------
  320m (8%)     300m (7%)       150Mi (1%)      150Mi (1%)

In your case either the CPU or memory allocation should be close to 100%.

Solution

  • Add more computing resources / nodes to your cluster (preferred)
  • Remove the resource requests from your pod(s), at the risk of overcommitting your resources