kubernetes(k 8 s)でmetrics-serverを導入する際に発生する問題
: kubernetes(k8s) metrics-server
metrics-server :k8s.gcr.io/metrics-server-amd64:v0.3.3
k8s :v1.14.1
The '--source' flag is unavailable right now (v0.3.0-alpha.1)
containers:
- name: metrics-server
image: k8s.gcr.io/metrics-server-amd64:v0.2.1
command:
- /metrics-server
- --source=kubernetes.summary_api:''?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250&insecure=true
- --requestheader-allowed-names=
metrics-serverのyamlファイルを適用した後、metrics-server-v 0を表示します.3.3-6 cbfd 7955 f-v 29 n 7状態はCrashLoopBackOff
[root@master metrics-server]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-fb8b8dccf-9bnvc 1/1 Running 3 20d
coredns-fb8b8dccf-n5bzb 1/1 Running 3 20d
etcd-master 1/1 Running 4 20d
kube-apiserver-master 1/1 Running 4 20d
kube-controller-manager-master 1/1 Running 5 20d
kube-proxy-cmmbw 1/1 Running 4 20d
kube-proxy-dnw9v 1/1 Running 4 20d
kube-proxy-s8zwl 1/1 Running 3 20d
kube-proxy-tt2vb 1/1 Running 4 20d
kube-scheduler-master 1/1 Running 4 20d
kubernetes-dashboard-5f7b999d65-n56td 1/1 Running 0 2d21h
metrics-server-v0.3.3-6cbfd7955f-v29n7 0/2 CrashLoopBackOff 1 16s
weave-net-6pv9w 2/2 Running 11 20d
weave-net-9dsxr 2/2 Running 10 20d
weave-net-lt8vv 2/2 Running 9 20d
weave-net-m2m8t 2/2 Running 11 20d
ログを表示すると、コンテナmetrics-serverに次のエラーが表示されます.
[root@master metrics-server]# kubectl logs metrics-server-v0.3.3-549cc669c7-gxvgw -c metrics-server -n kube-system
Flag --deprecated-kubelet-completely-insecure has been deprecated, This is rarely the right option, since it leaves kubelet communication completely insecure. If you encounter auth errors, make sure you've enabled token webhook auth on the Kubelet, and if you're in a test cluster with self-signed Kubelet certificates, consider using kubelet-insecure-tls instead.
I0531 04:36:16.382275 1 serving.go:312] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
I0531 04:36:16.963800 1 secure_serving.go:116] Serving securely on [::]:443
E0531 04:36:46.981720 1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:worker1: unable to fetch metrics from Kubelet worker1 (10.5.24.223): Get http://10.5.24.223:10255/stats/summary/: dial tcp 10.5.24.223:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:worker3: unable to fetch metrics from Kubelet worker3 (10.5.24.225): Get http://10.5.24.225:10255/stats/summary/: dial tcp 10.5.24.225:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:master: unable to fetch metrics from Kubelet master (10.5.24.222): Get http://10.5.24.222:10255/stats/summary/: dial tcp 10.5.24.222:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:worker2: unable to fetch metrics from Kubelet worker2 (10.5.24.224): Get http://10.5.24.224:10255/stats/summary/: dial tcp 10.5.24.224:10255: connect: connection refused]
E0531 04:37:16.978839 1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:worker1: unable to fetch metrics from Kubelet worker1 (10.5.24.223): Get http://10.5.24.223:10255/stats/summary/: dial tcp 10.5.24.223:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:master: unable to fetch metrics from Kubelet master (10.5.24.222): Get http://10.5.24.222:10255/stats/summary/: dial tcp 10.5.24.222:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:worker3: unable to fetch metrics from Kubelet worker3 (10.5.24.225): Get http://10.5.24.225:10255/stats/summary/: dial tcp 10.5.24.225:10255: connect: connection refused, unable to fully scrape metrics from source kubelet_summary:worker2: unable to fetch metrics from Kubelet worker2 (10.5.24.224): Get http://10.5.24.224:10255/stats/summary/: dial tcp 10.5.24.224:10255: connect: connection refused]
コンテナmetrics-server-nannyは次のエラーを報告しました.
[root@master metrics-server]# kubectl logs metrics-server-v0.3.3-55f94f8d76-ghlfb -n kube-system -c metrics-server-nanny
ERROR: logging before flag.Parse: I0531 06:20:40.721189 1 pod_nanny.go:65] Invoked by [/pod_nanny --config-dir=/etc/config --cpu={{ base_metrics_server_cpu }} --extra-cpu=0.5m --memory={{ base_metrics_server_memory }} --extra-memory={{ metrics_server_memory_per_node }}Mi --threshold=5 --deployment=metrics-server-v0.3.3 --container=metrics-server --poll-period=300000 --estimator=exponential --minClusterSize={{ metrics_server_min_cluster_size }}]
invalid argument "{{ metrics_server_min_cluster_size }}" for "--minClusterSize" flag: strconv.ParseUint: parsing "{{ metrics_server_min_cluster_size }}": invalid syntax
Usage of /pod_nanny:
--config-dir string Path of configuration containing base resource requirements. (default "MISSING")
--container string The name of the container to watch. This defaults to the nanny itself. (default "pod-nanny")
--cpu string The base CPU resource requirement.
--deployment string The name of the deployment being monitored. This is required.
--estimator string The estimator to use. Currently supported: linear, exponential (default "linear")
--extra-cpu string The amount of CPU to add per node.
--extra-memory string The amount of memory to add per node.
--extra-storage string The amount of storage to add per node. (default "0Gi")
--memory string The base memory resource requirement.
--minClusterSize uint The smallest number of nodes resources will be scaled to. Must be > 1. This flag is used only when an exponential estimator is used. (default 16)
--namespace string The namespace of the ward. This defaults to the nanny pod's own namespace. (default "kube-system")
--pod string The name of the pod to watch. This defaults to the nanny's own pod. (default "metrics-server-v0.3.3-55f94f8d76-ghlfb")
--poll-period int The time, in milliseconds, to poll the dependent container. (default 10000)
--storage string The base storage resource requirement. (default "MISSING")
--threshold int A number between 0-100. The dependent's resources are rewritten when they deviate from expected by more than threshold.
ソリューション:metrics-server-deploymentを変更します.yamlは以下の内容です
[root@master metrics-server]# cat metrics-server-deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: metrics-server
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metrics-server-config
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: EnsureExists
data:
NannyConfiguration: |-
apiVersion: nannyconfig/v1alpha1
kind: NannyConfiguration
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: metrics-server-v0.3.3
namespace: kube-system
labels:
k8s-app: metrics-server
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
version: v0.3.3
spec:
selector:
matchLabels:
k8s-app: metrics-server
version: v0.3.3
template:
metadata:
name: metrics-server
labels:
k8s-app: metrics-server
version: v0.3.3
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
spec:
priorityClassName: system-cluster-critical
serviceAccountName: metrics-server
containers:
- name: metrics-server
image: k8s.gcr.io/metrics-server-amd64:v0.3.3
command:
- /metrics-server
- --metric-resolution=30s
# These are needed for GKE, which doesn't support secure communication yet.
# Remove these lines for non-GKE clusters, and when GKE supports token-based auth.
#- --kubelet-port=10255
#- --deprecated-kubelet-completely-insecure=true
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP
#- --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP
ports:
- containerPort: 443
name: https
protocol: TCP
- name: metrics-server-nanny
image: k8s.gcr.io/addon-resizer:1.8.5
resources:
limits:
cpu: 100m
memory: 300Mi
requests:
cpu: 5m
memory: 50Mi
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: metrics-server-config-volume
mountPath: /etc/config
command:
- /pod_nanny
- --config-dir=/etc/config
#- --cpu={{ base_metrics_server_cpu }}
- --extra-cpu=0.5m
#- --memory={{ base_metrics_server_memory }}
#- --extra-memory={{ metrics_server_memory_per_node }}Mi
- --threshold=5
- --deployment=metrics-server-v0.3.3
- --container=metrics-server
- --poll-period=300000
- --estimator=exponential
# Specifies the smallest cluster (defined in number of nodes)
# resources will be scaled to.
- --minClusterSize=2
volumes:
- name: metrics-server-config-volume
configMap:
name: metrics-server-config
tolerations:
- key: "CriticalAddonsOnly"
operator: "Exists"
resource-readerを変更yamlは次のとおりです.
[root@master metrics-server]# cat resource-reader.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:metrics-server
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
- ""
resources:
- pods
- nodes
- nodes/stats
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- deployments
verbs:
- get
- list
- update
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:metrics-server
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:metrics-server
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
修正後に再applyに成功
[root@master metrics-server]# kubectl top nodes
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
master 336m 8% 2791Mi 36%
worker1 258m 6% 1583Mi 59%
worker2 172m 4% 1451Mi 54%
worker3 312m 7% 1672Mi 62%
[root@master metrics-server]# kubectl top pods -n kube-system
NAME CPU(cores) MEMORY(bytes)
coredns-fb8b8dccf-9bnvc 11m 16Mi
coredns-fb8b8dccf-n5bzb 9m 17Mi
etcd-master 39m 361Mi
kube-apiserver-master 116m 410Mi
kube-controller-manager-master 44m 66Mi
kube-proxy-cmmbw 2m 17Mi
kube-proxy-dnw9v 3m 23Mi
kube-proxy-s8zwl 5m 16Mi
kube-proxy-tt2vb 8m 22Mi
kube-scheduler-master 3m 18Mi
kubernetes-dashboard-5f7b999d65-n56td 1m 25Mi
metrics-server-v0.3.3-7856b88bf4-hstvq 3m 22Mi
weave-net-6pv9w 3m 114Mi
weave-net-9dsxr 3m 123Mi
weave-net-lt8vv 4m 105Mi
weave-net-m2m8t 5m 115Mi
参照先:https://github.com/kubernetes-incubator/metrics-server/issues/131
転載は明記してください:lampNick»kubernetes(k 8 s)の中でmetrics-serverを配置して出会った問題