kubernetesはk 8 s-prometheus-adapterに基づいてpodのhpa(自動伸縮)を実現する
113572 ワード
システム
kubernetesのバージョン
コードk 8 s-prometheus-adapterの取得
ブランチを
k 8 s-prometheus-adapterミラーの構築
k 8 s-prometheus-adapterミラーの表示
ミラーtagを変更し、プライベートミラーウェアハウスに指定
cm-adapter-serving-certsという名前のsecretを生成し、serving.crtとserving.key、実は証明書です
証明書生成はrequestheader-authenticationを参照してください
ここに証明書を生成するスクリプトがあります.参考にしてください.この証明書生成スクリプトにはgcc golang環境が必要です.gcc goをインストールします.
スクリプトの内容
証明書の生成
スクリプトによって生成されたコンテンツの表示
証明書の内容の表示
k 8 s-prometheus-adapter configmappを作成するには、この方法を使用します.
この方法も使えます
作成
配置
ミラー
変更後
kube-apiserverの起動パラメータの追加
kube-controller-managerの起動パラメータの追加
配置結果の表示
api-versionsの表示
この
次は検証です
CPU cpu_の取得usage
メモリベース
cpuベースcpu_アメリカの使用
問題はissuesですが、使用には影響しません.
end
リファレンス
配置方法
https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/v0.2.1/docs/walkthrough.md
[root@k8s-build ~]# uname -a
Linux k8s-build 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
kubernetesのバージョン
[root@master-47-35 custom-metrics-api]# kubectl version
Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0-1+feebbe2b16f546", GitCommit:"feebbe2b16f546ac289f7afdae40f1721e0656fe", GitTreeState:"clean", BuildDate:"2018-08-23T02:06:55Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.0-1+feebbe2b16f546", GitCommit:"feebbe2b16f546ac289f7afdae40f1721e0656fe", GitTreeState:"clean", BuildDate:"2018-08-23T02:06:55Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
コードk 8 s-prometheus-adapterの取得
yum install git -y
cd /root/gopath/src/
git clone https://github.com/DirectXMan12/k8s-prometheus-adapter.git
ブランチを
v0.2.1
に切り替え [root@k8s-build k8s-prometheus-adapter]# git checkout v0.2.1
Note: checking out 'v0.2.1'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD 3fc2dbd... Explicitly build on tags in Travis
[root@k8s-build k8s-prometheus-adapter]# git status
# v0.2.1
,
k 8 s-prometheus-adapterミラーの構築
[root@k8s-build k8s-prometheus-adapter]# make docker-build
cp deploy/Dockerfile /tmp/tmp.FbNVQfpHCg
cd /tmp/tmp.FbNVQfpHCg && sed -i "s|BASEIMAGE|busybox|g" Dockerfile
docker run -it -v /tmp/tmp.FbNVQfpHCg:/build -v /root/gopath/src/k8s-prometheus-adapter:/go/src/github.com/directxman12/k8s-prometheus-adapter -e GOARCH=amd64 golang:1.10 /bin/bash -c "\
CGO_ENABLED=0 go build -tags netgo -o /build/adapter github.com/directxman12/k8s-prometheus-adapter/cmd/adapter"
Unable to find image 'golang:1.10' locally
Trying to pull repository docker.io/library/golang ...
1.10: Pulling from docker.io/library/golang
05d1a5232b46: Pull complete
5cee356eda6b: Pull complete
89d3385f0fd3: Pull complete
80ae6b477848: Pull complete
94ebfeaaddf3: Pull complete
e132030a369d: Pull complete
c67c5750c788: Pull complete
Digest: sha256:322a64a16553516761f8e71eb94136140031e6a7fcfe794e579de14a7da21ee7
Status: Downloaded newer image for docker.io/golang:1.10
docker build -t directxman12/k8s-prometheus-adapter-amd64:latest /tmp/tmp.FbNVQfpHCg
Sending build context to Docker daemon 55.61 MB
Step 1/4 : FROM busybox
Trying to pull repository docker.io/library/busybox ...
latest: Pulling from docker.io/library/busybox
8c5a7da1afbc: Already exists
Digest: sha256:032ddd66f10483436e8a252e69fdfd20d0164e9953585c10d378183a0924db34
Status: Downloaded newer image for docker.io/busybox:latest
---> e1ddd7948a1c
Step 2/4 : COPY adapter /
---> a39120e24ddb
Removing intermediate container 6358600308cb
Step 3/4 : USER 1001:1001
---> Running in 9755e39bed35
---> 718d846e413b
Removing intermediate container 9755e39bed35
Step 4/4 : ENTRYPOINT /adapter
---> Running in 68554cb44316
---> cb309c2eb2f4
Removing intermediate container 68554cb44316
Successfully built cb309c2eb2f4
rm -rf /tmp/tmp.FbNVQfpHCg
k 8 s-prometheus-adapterミラーの表示
[root@k8s-build k8s-prometheus-adapter]# docker images | grep k8s-prometheus-adapter-amd64
directxman12/k8s-prometheus-adapter-amd64 latest cb309c2eb2f4 About a minute ago 56.8 MB
ミラーtagを変更し、プライベートミラーウェアハウスに指定
[root@k8s-build k8s-prometheus-adapter]# docker tag cb309c2eb2f4 harbor.enncloud.cn/enncloud/k8s-prometheus-adapter-amd64:v0.2.1
[root@k8s-build k8s-prometheus-adapter]# docker push harbor.enncloud.cn/enncloud/k8s-prometheus-adapter-amd64:v0.2.1
The push refers to a repository [harbor.enncloud.cn/enncloud/k8s-prometheus-adapter-amd64]
d4821deae84a: Pushed
f9d9e4e6e2f0: Pushed
v0.2.1: digest: sha256:32ead17195963566630d9a8286773ddff92f9ccc126cf9a6e2cdcf686d163aef size: 739
cm-adapter-serving-certsという名前のsecretを生成し、serving.crtとserving.key、実は証明書です
証明書生成はrequestheader-authenticationを参照してください
ここに証明書を生成するスクリプトがあります.参考にしてください.この証明書生成スクリプトにはgcc golang環境が必要です.gcc goをインストールします.
yum install go gcc -y
スクリプトの内容
[root@k8s-build certs]# cat gencerts.sh
#!/usr/bin/env bash
# Detect if we are on mac or should use GNU base64 options
case $(uname) in
Darwin)
b64_opts='-b=0'
;;
*)
b64_opts='--wrap=0'
esac
go get -v -u github.com/cloudflare/cfssl/cmd/...
export PATH=$GOPATH/bin:$PATH
export PURPOSE=metrics
openssl req -x509 -sha256 -new -nodes -days 365 -newkey rsa:2048 -keyout ${PURPOSE}-ca.key -out ${PURPOSE}-ca.crt -subj "/CN=ca"
echo '{"signing":{"default":{"expiry":"43800h","usages":["signing","key encipherment","'${PURPOSE}'"]}}}' > "${PURPOSE}-ca-config.json"
export SERVICE_NAME=custom-metrics-apiserver
export ALT_NAMES='"custom-metrics-apiserver.custom-metrics","custom-metrics-apiserver.custom-metrics.svc"'
echo "{\"CN\":\"${SERVICE_NAME}\", \"hosts\": [${ALT_NAMES}], \"key\": {\"algo\": \"rsa\",\"size\": 2048}}" | \
cfssl gencert -ca=metrics-ca.crt -ca-key=metrics-ca.key -config=metrics-ca-config.json - | cfssljson -bare apiserver
cat < cm-adapter-serving-certs.yaml
apiVersion: v1
kind: Secret
metadata:
name: cm-adapter-serving-certs
namespace: custom-metrics
data:
serving.crt: $(base64 ${b64_opts} < apiserver.pem)
serving.key: $(base64 ${b64_opts} < apiserver-key.pem)
EOF
証明書の生成
[root@k8s-build certs]# ./gencerts.sh
github.com/cloudflare/cfssl (download)
Generating a 2048 bit RSA private key
.+++
......+++
writing new private key to 'metrics-ca.key'
-----
2018/08/25 20:36:58 [INFO] generate received request
2018/08/25 20:36:58 [INFO] received CSR
2018/08/25 20:36:58 [INFO] generating key: rsa-2048
2018/08/25 20:36:59 [INFO] encoded CSR
2018/08/25 20:36:59 [INFO] signed certificate with serial number 573231977150840198182111747492918974864906787988
スクリプトによって生成されたコンテンツの表示
[root@k8s-build certs]# ls -l
32
-rw-r--r-- 1 root root 1066 8 25 20:36 apiserver.csr
-rw------- 1 root root 1679 8 25 20:36 apiserver-key.pem
-rw-r--r-- 1 root root 1277 8 25 20:36 apiserver.pem
-rw-r--r-- 1 root root 4053 8 25 20:36 cm-adapter-serving-certs.yaml
-rwxr-xr-x 1 root root 1198 8 25 20:36 gencerts.sh
-rw-r--r-- 1 root root 94 8 25 20:36 metrics-ca-config.json
-rw-r--r-- 1 root root 1074 8 25 20:36 metrics-ca.crt
-rw-r--r-- 1 root root 1704 8 25 20:36 metrics-ca.key
証明書の内容の表示
[root@master-47-35 custom-metrics]# cat cm-adapter-serving-certs.yaml
apiVersion: v1
data:
serving.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURHRENDQWdDZ0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFmTVIwd0d3WURWUVFEREJSc2IyTmgKYkdodmMzUkFNVFV4TXpjME5qZzBNREFlRncweE56RXlNakF3TlRFME1EQmFGdzB4T0RFeU1qQXdOVEUwTURCYQpNQjh4SFRBYkJnTlZCQU1NRkd4dlkyRnNhRzl6ZEVBeE5URXpOelEyT0RRd01JSUJJakFOQmdrcWhraUc5dzBCCkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQW9mZE4zd1RqVldlZFo2YlBRN3ZrMTNPeUkwMEVwOGcvYzlTTCtneWMKQ3RHK3hYSkhrVlN4eW1zVlhsNnF4T2tWTWkrL2JqRFVqc1VsSnY5a2JjZkhHemYwYmxiQWdocVhvQ20ycjkvRwp0TVNlczlTL0NER1FNQUhPbkxsemdjVkxVQ1kyUVVXdS9HZDVwQU96dkJSUVBWczNBNUNZcDMzS0grTFVTTExOCnVNMjcvSkVnclUwczdBQ29ZaDk3KzQrSU1qb1pmbHRiK0JOQzRaU3BvemhQZ0EwQWhXRXl0SXkrRDdVWWdIR2cKNm8ybWJpTkV1QUNrQmk5TmFITm8zdGdieVN3OGQzYU9vd3JQQ3VlNEV0UTNIZGlNMHB5WUtuSUhsTFhyNjA1Sgpra25ENWs0OWFoaktQUFJhTlpjNlZpZlEvTzNiekVnNUwvVld6K2lneFJrSyt3SURBUUFCbzE4d1hUQU9CZ05WCkhROEJBZjhFQkFNQ0FxUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0V3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBbEJnTlZIUkVFSGpBY2dnbHNiMk5oYkdodmMzU0NDV3h2WTJGc2FHOXpkSWNFZndBQUFUQU5CZ2txaGtpRwo5dzBCQVFzRkFBT0NBUUVBWERVV3ozVW0zVFg4VFJCVmQ1VVF5Z1ZhKzloMlRtVWJMU0JpdlBURDVRTWFBZUJzCkRYQU0vbTZrWXZIcFlTTi9MOWhraWd1QzhTazdlVFZqeWZabjlsYmNoSVV1NytoYys1UThJTDZVb0VaQ2cxVHQKNDNlYzBhZlc1YUU2bm5lejVhMWs3UDFhY0xhL1ZFUzVWM25wUmg2YUl4a0xVVHNMaUFsdGRGcURpeTF4aVlORgorL2pWMGFHRXhWWWJMbFVQMkIxaTlQODZneGFjNkJRTCtSQTNZZjFISWl5ZkliemMrbjV4Z2U1T0NIbC9XVjV1CkIzTHdZdkh1TGdpTDNZbyswN0gycWZ3MlNIUmZxdlJHdlpmS2tnWmcrTVQrZElYR3ZTTE0ybUhhdElxQnNXZEwKS1VFZnorREdYbzBIam1LTmhuWThRM0kwN1IxY29ta05KUFMxL3c9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
serving.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb2dJQkFBS0NBUUVBb2ZkTjN3VGpWV2VkWjZiUFE3dmsxM095STAwRXA4Zy9jOVNMK2d5Y0N0Ryt4WEpICmtWU3h5bXNWWGw2cXhPa1ZNaSsvYmpEVWpzVWxKdjlrYmNmSEd6ZjBibGJBZ2hxWG9DbTJyOS9HdE1TZXM5Uy8KQ0RHUU1BSE9uTGx6Z2NWTFVDWTJRVVd1L0dkNXBBT3p2QlJRUFZzM0E1Q1lwMzNLSCtMVVNMTE51TTI3L0pFZwpyVTBzN0FDb1loOTcrNCtJTWpvWmZsdGIrQk5DNFpTcG96aFBnQTBBaFdFeXRJeStEN1VZZ0hHZzZvMm1iaU5FCnVBQ2tCaTlOYUhObzN0Z2J5U3c4ZDNhT293clBDdWU0RXRRM0hkaU0wcHlZS25JSGxMWHI2MDVKa2tuRDVrNDkKYWhqS1BQUmFOWmM2VmlmUS9PM2J6RWc1TC9WV3oraWd4UmtLK3dJREFRQUJBb0lCQURUci82L05neEtXNXFIRgpxWmV5UDFYKzRjZ1RGR0JXV0JWYTc5bm1ZTjVRSm91Umliejg5WkwwWm1rTmViTVpqZWwySEpDK3Q5Vm1xWHNTCm5CZ2EzUDRSekhaNU5qSE15cHAweHRLMHhkS3ZKRVFTVWRFMWtLMS9DYW5BRnZQaUtYZW81TFZ6TlhnWHYzRk8KSEY1cUxNVy9sanc5OVFOVllNUUdjK0xGT2Q4V0pRRm9KZVh3bFBKdC9vK3NWUCtHK3dtYmp3VUNtc3ZqTFc5NAoxbS9XeWtwVlMrMCt3QVRJTGJCeTJqUUVtK3hwZk1IT0xhU01Hem9oUnEySHFvY1d2NE10OWI4bmFPNURyQ3dCCmZIQnIxd3VtL3duWGVESUFQay8yVVYxSllzdDQveVFjSENUS1V0Qzh3WFd6V3RabEE3dFNmM1huNUM0MXRhakoKaTIxMzNuRUNnWUVBd1BUOVRUS3R4NzZHYVovZE5WWStLY1ExYnplQlA5NlI0SFp0M1d4NllUbEJZZ2oxQWg4dQpZVGJCYmhYdDExcTQ0b0h4eXk0eThCVC9xWU55dWhIUGJjMExCYW4zamtwRVZEZ3VaempMSHc3YUUwTlVkNFNDCmFzd0FjTGlnMEdiTU1RMVJnZndHdVpOc3ZXZ2wrVXZWdVB3SVRKUzBUYkdQeVVGTnpZSmlsUjBDZ1lFQTF1STMKaFB1VjBoOGUyZEVHT3g5L0FXb1hCU3lhajNVZ2xTV2VnNTNBOFJ1OU9POTl5OGxHSXZxUDdsVXpub01xZzlwMwpEdXF3TFdCSnQwYU11SG1yZVF6RzhiME9ieG1UTmwreTdUaTIwOU9nUTBaZ0ZhSFhtbng2UStaWlovNFd0UXlMCmxzUUxPMDNqcjUwb3dQV3FJSnRmdEVXckZsYXBCd01IdFBxY0hQY0NnWUJVT1J0dlhOV1p3RUh2VE1aRGg3ZHUKaDZSRFBWai9SZWtxcnNiWnhZWE5kaTVoMmtNSGdCREFnaVFBSnpZL3IrdlBncVRGNlE1MjJOT1hiRGJoblBDTApudHBXeUkrejNocmhIL3l4TllzRHJYbm11cGJkN0VyNHpyN0tkT3hNeXZJSjN5UmF0bVZCRDJhQmJqcFZESkxuClFhdGxzZXJabHMyS0xTVUd0UWpPeVFLQmdDTUVpbmVwYUNqdDJXRTFwd1pMZzY3L1V2RU02OE1sRzBJeTl6b0wKb3oybmY1YWZzS2hkRElqMnZDM2l4SjY0VVYxYlpMdVJrMVJSdExHaFZaMWFkbzhyZmVBM0p2c0Q5L0dNZHVRaApzZW1qT3lLN2ZUMEpPdEY2SjVLWkgwakxKM2dwcXpYT2pPWUZNMjJTOXVrN0g1Wmd1NS90R294eFl1bzVuQy9xCkJ6YlJBb0dBVHUwc2Rnb2dxL2RHdDFIVjNmZnhyVGhVazBMdVZxM0x4Ni9mN0tXWGN4b21qUWlQc0RZUTVkU3IKb21Kb2wvVUxWUnRQbGJtcmZKN1VpcHBlNXh1eUFNQUVocldIeG15SlR2TWZQSks3UGs4eHgrcUlSUGd0YklobwpWcjN5ZFoyYnRka2h4OXo5dGY4WXJ5RkxqNnNIWTY4V1Z2T3hETkcwYTZPYUhWWFJJMUE9Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
kind: Secret
metadata:
name: cm-adapter-serving-certs
namespace: custom-metrics
k 8 s-prometheus-adapter configmappを作成するには、この方法を使用します.
kubectl create secret generic cm-adapter-serving-certs --from-file=serving.crt=apiserver.crt --from-file=serving.key=apiserver.key -n custom-metrics
この方法も使えます
kubectl create -f cm-adapter-serving-certs.yaml
作成
custom-metrics namespace
kubectl create namespace custom-metrics
配置
ミラー
custom-metrics-apiserver-deployment.yaml
のミラーの変更cd /opt/ssl/custom-metrics/k8s-prometheus-adapter/deploy/manifests
custom-metrics-apiserver-deployment.yaml
image: directxman12/k8s-prometheus-adapter-amd64
to
image: harbor.enncloud.cn/enncloud/k8s-prometheus-adapter-amd64:v0.2.1
- --prometheus-url=http://prometheus.prom.svc:9090/
to
- --prometheus-url=http://prometheus-monitor.kube-system:9090/
- --config=/default-config.yaml
to
- --config=/etc/adapter/config.yaml
- --requestheader-client-ca-file=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
変更後
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: custom-metrics-apiserver
name: custom-metrics-apiserver
namespace: custom-metrics
spec:
replicas: 1
selector:
matchLabels:
app: custom-metrics-apiserver
template:
metadata:
labels:
app: custom-metrics-apiserver
name: custom-metrics-apiserver
spec:
serviceAccountName: custom-metrics-apiserver
containers:
- name: custom-metrics-apiserver
image: directxman12/k8s-prometheus-adapter-amd64
args:
- /adapter
- --secure-port=6443
- --tls-cert-file=/var/run/serving-cert/serving.crt
- --tls-private-key-file=/var/run/serving-cert/serving.key
- --logtostderr=true
- --prometheus-url=http://prometheus-monitor.kube-system:9090/
- --metrics-relist-interval=1m
- --requestheader-client-ca-file=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- --v=10
- --config=/etc/adapter/config.yaml
ports:
- containerPort: 6443
volumeMounts:
- mountPath: /var/run/serving-cert
name: volume-serving-cert
readOnly: true
- mountPath: /etc/adapter/
name: config
readOnly: true
- mountPath: /tmp
name: tmp-vol
volumes:
- name: volume-serving-cert
secret:
secretName: cm-adapter-serving-certs
- name: config
configMap:
name: adapter-config
- name: tmp-vol
emptyDir: {}
kube-apiserverの起動パラメータの追加
--requestheader-client-ca-file=/etc/kubernetes/ssl/ca.pem \
--requestheader-allowed-names=aggregator \
--requestheader-extra-headers-prefix=X-Remote-Extra- \
--requestheader-group-headers=X-Remote-Group \
--requestheader-username-headers=X-Remote-User \
--proxy-client-cert-file=/etc/kubernetes/ssl/aggregator-proxy.pem \
--proxy-client-key-file=/etc/kubernetes/ssl/aggregator-proxy-key.pem \
--enable-aggregator-routing=true \
kube-controller-managerの起動パラメータの追加
"--horizontal-pod-autoscaler-use-rest-clients",
"--horizontal-pod-autoscaler-sync-period=60s",
cd /opt/ssl/custom-metrics/k8s-prometheus-adapter/deploy/manifests
kubectl create -f .
配置結果の表示
[root@master-47-35 custom-metrics]# kubectl get all -n custom-metrics
NAME READY STATUS RESTARTS AGE
pod/custom-metrics-apiserver-5d8dd96b5-8s9rq 1/1 Running 0 16h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/custom-metrics-apiserver ClusterIP 10.254.40.72 443/TCP 16h
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/custom-metrics-apiserver 1 1 1 1 16h
NAME DESIRED CURRENT READY AGE
replicaset.apps/custom-metrics-apiserver-5d8dd96b5 1 1 1 16h
api-versionsの表示
[root@master-47-35 k8s-prometheus-adapter]# kubectl api-versions
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1
apiregistration.k8s.io/v1beta1
apps/v1
apps/v1beta1
apps/v1beta2
authentication.k8s.io/v1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1
authorization.k8s.io/v1beta1
autoscaling/v1
autoscaling/v2beta1
batch/v1
batch/v1beta1
certificates.k8s.io/v1beta1
custom.metrics.k8s.io/v1beta1
events.k8s.io/v1beta1
extensions/v1beta1
monitoring.coreos.com/v1
networking.k8s.io/v1
policy/v1beta1
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1
scheduling.k8s.io/v1beta1
storage.k8s.io/v1
storage.k8s.io/v1beta1
v1
この
custom.metrics.k8s.io/v1beta1
apiが存在することを確認しますroot@master-47-35 ~]# kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1"
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"custom.metrics.k8s.io/v1beta1","resources":[{"name":"pods/fs_usage_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_memory_reservation_limit_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_io_time","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_reads_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_writes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/network_tcp_usage","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_sector_writes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_writes_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_memory_limit_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/start_time_seconds","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_cpu_shares","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_memory_reservation_limit_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_read","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_cfs_throttled_periods","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_load_average_10s","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_swap","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_failures","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/start_time_seconds","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_max_usage_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_cpu_period","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_schedstat_run","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_io_time_weighted","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/network_tcp_usage","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_io_current","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_cpu_quota","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_cfs_throttled","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_cfs_periods","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_io_current","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_usage_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_reads_merged","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_reads","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_writes_merged","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_failures","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_user","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_write","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_cfs_throttled_periods","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_schedstat_run_periods","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_load_average_10s","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/last_seen","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_cpu_period","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_system","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_read","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_schedstat_run_periods","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_sector_reads","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_rss","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_rss","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_working_set_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_inodes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_reads_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_writes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_cache","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_cpu_shares","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_reads_merged","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_sector_reads","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_usage_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_failcnt","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_memory_swap_limit_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/spec_memory_swap_limit_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_schedstat_run","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_usage","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/last_seen","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_cache","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_writes_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_limit_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_max_usage_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_working_set_bytes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_schedstat_runqueue","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_system","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_cfs_periods","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_sector_writes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_usage","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_writes_merged","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/network_udp_usage","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_swap","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/cpu_cfs_throttled","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_user","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_inodes","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/network_udp_usage","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_memory_limit_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/tasks_state","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_io_time","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_write","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/memory_usage_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/spec_cpu_quota","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_inodes_free","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_limit_bytes","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/memory_failcnt","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/tasks_state","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/cpu_schedstat_runqueue","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_io_time_weighted","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/fs_reads","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/fs_inodes_free","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]}]}
次は検証です
CPU cpu_の取得usage
[root@master-47-35 ~]# kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/cpu_usage"
{"kind":"MetricValueList","apiVersion":"custom.metrics.k8s.io/v1beta1","metadata":{"selfLink":"/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/%2A/cpu_usage"},"items":[{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-qg2gj","apiVersion":"/__internal"},"metricName":"cpu_usage","timestamp":"2018-09-12T14:49:30Z","value":"0"},{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-t4xf2","apiVersion":"/__internal"},"metricName":"cpu_usage","timestamp":"2018-09-12T14:49:30Z","value":"0"},{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-w26gb","apiVersion":"/__internal"},"metricName":"cpu_usage","timestamp":"2018-09-12T14:49:30Z","value":"0"}]}
memory_usage_bytes
[root@master-47-35 ~]# kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/memory_usage_bytes"
{"kind":"MetricValueList","apiVersion":"custom.metrics.k8s.io/v1beta1","metadata":{"selfLink":"/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/%2A/memory_usage_bytes"},"items":[{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-qg2gj","apiVersion":"/__internal"},"metricName":"memory_usage_bytes","timestamp":"2018-09-12T14:52:46Z","value":"3547136"},{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-t4xf2","apiVersion":"/__internal"},"metricName":"memory_usage_bytes","timestamp":"2018-09-12T14:52:46Z","value":"4435968"},{"describedObject":{"kind":"Pod","namespace":"default","name":"nginx-dm-fff68d674-w26gb","apiVersion":"/__internal"},"metricName":"memory_usage_bytes","timestamp":"2018-09-12T14:52:46Z","value":"6172672"}]}
v1beta1.custom.metrics.k8s.io
が存在するかどうかを確認[root@master-47-35 ~]# kubectl get apiservice v1beta1.custom.metrics.k8s.io -o yaml
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
creationTimestamp: 2018-09-12T08:57:08Z
deletionGracePeriodSeconds: 0
deletionTimestamp: 2018-09-12T10:34:21Z
finalizers:
- foregroundDeletion
name: v1beta1.custom.metrics.k8s.io
resourceVersion: "3189364"
selfLink: /apis/apiregistration.k8s.io/v1/apiservices/v1beta1.custom.metrics.k8s.io
uid: d470a606-b669-11e8-aec0-5254e98192ae
spec:
group: custom.metrics.k8s.io
groupPriorityMinimum: 100
insecureSkipTLSVerify: true
service:
name: custom-metrics-apiserver
namespace: custom-metrics
version: v1beta1
versionPriority: 100
status:
conditions:
- lastTransitionTime: 2018-09-12T10:14:40Z
message: all checks passed
reason: Passed
status: "True"
type: Available
メモリベース
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
annotations:
autoscaling.alpha.kubernetes.io/conditions: '[{"type":"AbleToScale","status":"True","lastTransitionTime":"2018-09-13T02:23:13Z","reason":"SucceededGetScale","message":"the
HPA controller was able to get the target''s current scale"},{"type":"ScalingActive","status":"False","lastTransitionTime":"2018-09-13T02:24:13Z","reason":"ScalingDisabled","message":"scaling
is disabled since the replica count of the target is zero"},{"type":"ScalingLimited","status":"True","lastTransitionTime":"2018-09-13T02:03:10Z","reason":"TooManyReplicas","message":"the
desired replica count is more than the maximum replica count"}]'
autoscaling.alpha.kubernetes.io/metrics: '[{"type":"Pods","pods":{"metricName":"memory_usage_bytes","targetAverageValue":"1M"}}]'
creationTimestamp: 2018-09-13T02:01:10Z
name: example2
namespace: demo
resourceVersion: "3299536"
selfLink: /apis/autoscaling/v1/namespaces/demo/horizontalpodautoscalers/example2
uid: e2be2af9-b6f8-11e8-bf72-5254e98192ae
spec:
maxReplicas: 2
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1beta1
kind: Deployment
name: example2
status:
currentReplicas: 0
desiredReplicas: 0
lastScaleTime: 2018-09-13T02:17:13Z
cpuベースcpu_アメリカの使用
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
annotations:
autoscaling.alpha.kubernetes.io/conditions: '[{"type":"AbleToScale","status":"True","lastTransitionTime":"2018-09-13T02:32:34Z","reason":"ReadyForNewScale","message":"the
last scale time was sufficiently old as to warrant a new scale"},{"type":"ScalingActive","status":"True","lastTransitionTime":"2018-09-13T02:23:33Z","reason":"ValidMetricFound","message":"the
HPA was able to successfully calculate a replica count from pods metric cpu_usage"},{"type":"ScalingLimited","status":"True","lastTransitionTime":"2018-09-13T02:23:33Z","reason":"TooManyReplicas","message":"the
desired replica count is more than the maximum replica count"}]'
autoscaling.alpha.kubernetes.io/current-metrics: '[{"type":"Pods","pods":{"metricName":"cpu_usage","currentAverageValue":"322m"}}]'
autoscaling.alpha.kubernetes.io/metrics: '[{"type":"Pods","pods":{"metricName":"cpu_usage","targetAverageValue":"30m"}}]'
creationTimestamp: 2018-09-13T02:21:33Z
name: example
namespace: demo
resourceVersion: "3301437"
selfLink: /apis/autoscaling/v1/namespaces/demo/horizontalpodautoscalers/example
uid: bbc859b6-b6fb-11e8-bf72-5254e98192ae
spec:
maxReplicas: 4
minReplicas: 2
scaleTargetRef:
apiVersion: apps/v1beta1
kind: Deployment
name: example
status:
currentReplicas: 4
desiredReplicas: 4
lastScaleTime: 2018-09-13T02:26:34Z
問題はissuesですが、使用には影響しません.
end
リファレンス
配置方法
https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/v0.2.1/docs/walkthrough.md