kubernetes pod NodeLost分析

5856 ワード

問題の説明
[root@master-64 ~]# kubectl get pods -n kube-system -owide| grep -v Running
NAME                                        READY     STATUS             RESTARTS   AGE       IP                NODE
calico-node-cnrvm                           2/2       NodeLost           0          48d       10.39.1.44        proxy-44
filebeat-k7svm                              1/1       NodeLost           0          15d       192.168.159.0     proxy-44
kube-proxy-oop2g                            1/1       NodeLost           0          62d       10.39.1.44        proxy-44
kubectl-qlj27                               1/1       NodeLost           0          15d       192.168.159.3     proxy-44
prometheus-tim-3864503240-ng4h0             0/1       CrashLoopBackOff   485        1d        192.168.88.70     slave-95
service-proxy-clq2t                         1/1       NodeLost           0          2h        10.39.1.44        proxy-44
[root@master ~]# kubectl describe pod calico-node-cnrvm -n kube-system
...
Node-Selectors: calico=2.5.1
Tolerations:    node.alpha.kubernetes.io/notReady:NoExecute
        node.alpha.kubernetes.io/unreachable:NoExecute
 /proc/2005/net/dev: no such file or directory
I0119 13:53:02.039754    2029 helpers.go:101] Unable to get network stats from pid 16385: couldn't read network stats: failure opening /proc/16385/net/dev: open /proc/16385/net/dev: no such file or directory

...
I0119 13:53:10.273923    2029 reconciler.go:299] MountVolume operation started for volume "kubernetes.io/secret/aee483dc-f6c3-11e7-9d36-5254b24cbf5e-default-token-qu72f" (spec.Name: "default-token-qu72f") to pod "aee483dc-f6c3-11e7-9d36-5254b24cbf5e" (UID: "aee483dc-f6c3-11e7-9d36-5254b24cbf5e"). Volume is already mounted to pod, but remount was requested.

...
I0119 13:54:33.846820    2029 conversion.go:133] failed to handle multiple devices for container. Skipping Filesystem stats
I0119 13:54:33.846949    2029 conversion.go:133] failed to handle multiple devices for container. Skipping Filesystem stats

問題を解決する
再起動命令を実行すればよい
docker restart containerID

END