How to restart kubernetes nodes?

nodes kubernetes
user_mda picture user_mda · Nov 12, 2015 · Viewed 135.6k times · Source

The status of nodes is reported as unknown

"conditions": [
          {
            "type": "Ready",
            "status": "Unknown",
            "lastHeartbeatTime": "2015-11-12T06:03:19Z",
            "lastTransitionTime": "2015-11-12T06:04:03Z",
            "reason": "Kubelet stopped posting node status."
          }

whle kubectl get nodes return a NOTReady status. What does this imply and how to fix this?

Answer

CHENJIAN picture CHENJIAN · Dec 1, 2016

Get nodes

kubectl get nodes

Result:

NAME            STATUS     AGE
192.168.1.157   NotReady   42d
192.168.1.158   Ready      42d
192.168.1.159   Ready      42d

Describe node

Here is a NotReady on the node of 192.168.1.157. Then debugging this notready node, and you can read offical documents - Application Introspection and Debugging.

kubectl describe node 192.168.1.157

Partial Result:

Conditions:
Type          Status          LastHeartbeatTime                       LastTransitionTime                      Reason                  Message
----          ------          -----------------                       ------------------                      ------                  -------
OutOfDisk     Unknown         Sat, 28 Dec 2016 12:56:01 +0000         Sat, 28 Dec 2016 12:56:41 +0000         NodeStatusUnknown       Kubelet stopped posting node status.
Ready         Unknown         Sat, 28 Dec 2016 12:56:01 +0000         Sat, 28 Dec 2016 12:56:41 +0000         NodeStatusUnknown       Kubelet stopped posting node status.

There is a OutOfDisk on my node, then Kubelet stopped posting node status. So, I must free some disk space, using the command of df on my Ubuntu14.04 I can check the details of memory, and using the command of docker rmi image_id/image_name under the role of su I can remove the useless images.

Login in node

Login in 192.168.1.157 by using ssh, like ssh [email protected], and switch to the 'su' by sudo su;

Restart kubelet

/etc/init.d/kubelet restart

Result:

stop: Unknown instance: 
kubelet start/running, process 59261

Get nodes again

On the master:

kubectl get nodes

Result:

NAME            STATUS    AGE
192.168.1.157   Ready     42d
192.168.1.158   Ready     42d
192.168.1.159   Ready     42d

Ok, that node works fine.

Here is a reference: Kubernetes

Related questions

What is the difference between tree depth and height?

This is a simple question from algorithms theory. The difference between them is that in one case you count number of nodes and in other number of edges on the shortest path between root and concrete node. Which is which?

Read More
Why is NULL undeclared?

I have a problem with this struct contructor when I try to compile this code: typedef struct Node { Node( int data ) // { this->data = data; previous = NULL; // Compiler indicates here next = NULL; } int data; Node* previous; Node* next; } NODE; when …

Read More
Node.js server that accepts POST requests

I'm trying to allow javascript to communicate with a Node.js server. POST request (web browser) var http = new XMLHttpRequest(); var params = "text=stuff"; http.open("POST", "http://someurl.net:8080", true); http.setRequestHeader("Content-type", "application/x-www-form-urlencoded"); http.setRequestHeader("Content-length", params.…

Read More