バイナリ配備kubernetes 1.7.4

25182 ワード
配置構造
  • 172.16.7.71 Masterノード
  • 172.16.7.72-73 nodeノード
  • 172.16.7.71、72、73 ETCDクラスタ
    • ETCD
    1、etcd、etcdctlを/usr/binディレクトリにコピーする
    2、新規/usr/lib/systemd/system/etcd.service 
    [Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\""
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

    3、新規プロファイル/etc/etcd/etcd.conf 
     [member]
ETCD_NAME=etcd3
ETCD_DATA_DIR="/var/lib/etcd/etcd3" 
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="http://172.16.7.73:2380"
ETCD_LISTEN_CLIENT_URLS="http://172.16.7.73:2379,http://127.0.0.1:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.7.73:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd1=http://172.16.7.71:2380,etcd2=http://172.16.7.72:2380,etcd3=http://172.16.7.73:2380"
ETCD_INITIAL_CLUSTER_STATE="new33"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster33"
ETCD_ADVERTISE_CLIENT_URLS="http://172.16.7.73:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#ETCD_AUTO_COMPACTION_RETENTION="0"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_AUTO_TLS="false"
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#ETCD_PEER_AUTO_TLS="false"
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""

    注意ETCD_NAME、ETCD_DATA_DIRはノードごとに構成が異なり、ETCD_INITIAL_CLUSTER_STATE、ETCD_INITIAL_CLUSTER_TOKENはクラスタごとに異なる構成が望ましい
    4、systemctl daemon-reloadそれからsystemctl start etcdでいいです.複数のノードが同時に実行するように注意してください.そうしないと、クラスタの初期化に失敗します.
    5、検証
    [root@localhost ~]# etcdctl member list
855390a4542535be: name=etcd3 peerURLs=http://172.16.7.73:2380 clientURLs=http://172.16.7.73:2379 isLeader=false
973d9cab4aa5f1ba: name=etcd1 peerURLs=http://172.16.7.71:2380 clientURLs=http://172.16.7.71:2379 isLeader=true
9ecaf535a6810487: name=etcd2 peerURLs=http://172.16.7.72:2380 clientURLs=http://172.16.7.72:2379 isLeader=false

    master->apiServer
    1、バイナリファイルを/usr/local/binディレクトリにコピーし、ログディレクトリ/var/logs/kubernetesを作成する
    2、プロファイル/usr/lib/systemd/system/kube-apiserverを追加する.service 
    [Unit]
Description=Kubernetes API Service
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target
After=etcd.service

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/apiserver
ExecStart=/usr/local/bin/kube-apiserver             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_ETCD_SERVERS             $KUBE_API_ADDRESS             $KUBE_API_PORT             $KUBELET_PORT             $KUBE_ALLOW_PRIV             $KUBE_SERVICE_ADDRESSES             $KUBE_ADMISSION_CONTROL             $KUBE_API_ARGS
Restart=on-failure
Type=notify
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

    3、プロファイル/etc/kubernetes/configの作成
    ###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
#   kube-apiserver.service
#   kube-controller-manager.service
#   kube-scheduler.service
#   kubelet.service
#   kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=false  --log-dir=/var/log/kubernetes"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=true"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://172.16.7.71:8080"

    4、プロファイル/etc/kubernetes/apiserverの作成
    ##
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
##

# The address on the local server to listen to.
KUBE_API_ADDRESS="--advertise-address=172.16.7.1 --insecure-bind-address=0.0.0.0"

# The port on the local server to listen on.
KUBE_API_PORT="--port=8080"

# Port minions listen on
KUBELET_PORT="--kubelet-port=10250"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://172.16.7.1:2379,http://172.16.7.2:2379,http://172.16.7.3:2379"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.96.0.0/12"

# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota"

# Add your own!
KUBE_API_ARGS="--storage-backend=etcd2 --enable-swagger-ui=true --apiserver-count=3 --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --audit-log-path=/var/lib/audit.log --event-ttl=1h"

    5、kubectlバイナリファイルを/usr/binディレクトリにコピーする
    6、kube-apiserverを起動する
    # systemctl daemon-reload
# systemctl enable kube-apiserver
# systemctl start kube-apiserver

    7、検証
    kubectl get componentstatuses
    [root@localhost kubernetes]# curl 172.16.7.71:8080
{
  "paths": [
    "/api",
    。。。。。。
    "/version"
  ]
}

    master->controller-manager
    1、バイナリファイルを/usr/local/binディレクトリにコピーする
    2、ファイルの作成/usr/lib/systemd/system/kube-controller-manager.service 
    Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/controller-manager
ExecStart=/usr/local/bin/kube-controller-manager             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_CONTROLLER_MANAGER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

    3、プロファイル/etc/kubernetes/controller-managerの作成
    ###
# The following values are used to configure the kubernetes controller-manager

# defaults from config and apiserver should be adequate

# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--master= --address=127.0.0.1 --service-cluster-ip-range=10.96.0.0/12 --cluster-name=kubernetes --leader-elect=true"

    4、起動
    # systemctl daemon-reload
# systemctl enable kube-apiserver
# systemctl start kube-apiserver

    master->scheduler
    1、バイナリファイルを/uar/local/binディレクトリにコピーする
    2、プロファイル/usr/lib/systemd/system/kube-schedulerを作成する.service 
    [Unit]
Description=Kubernetes Scheduler Plugin
Documentation=https://github.com/GoogleCloudPlatform/kubernetes

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/scheduler
ExecStart=/usr/local/bin/kube-scheduler             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_SCHEDULER_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

    3、プロファイル/etc/kubernetes/schedulerの作成
    ###
# kubernetes scheduler config

# default config should be adequate

# Add your own!
KUBE_SCHEDULER_ARGS="--leader-elect=true --address=127.0.0.1"

    4、起動
    # systemctl daemon-reload
# systemctl enable kube-scheduler
# systemctl start kube-scheduler

    5、検証
    [root@localhost kubernetes]# kubectl get componentstatuses
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
etcd-1               Healthy   {"health": "true"}   
etcd-2               Healthy   {"health": "true"}

    node->kubelet
    1、kubeletを/usr/local/binに入れ、dockerがインストール済みであることを確認する(overlayfs駆動が望ましい)
    2、プロファイル/usr/lib/systemd/system/kubeletを追加する.service 
    [Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service

[Service]
WorkingDirectory=/var/lib/kubelet
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/local/bin/kubelet             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBELET_API_SERVER             $KUBELET_ADDRESS             $KUBELET_PORT             $KUBELET_HOSTNAME             $KUBE_ALLOW_PRIV             $KUBELET_POD_INFRA_CONTAINER             $KUBELET_ARGS
Restart=on-failure

[Install]
WantedBy=multi-user.target

    3、ディレクトリmkdir-p/etc/kubernetes//var/lib/kubelet//var/logs/kubernetesの作成
    4、プロファイル/etc/kubernetes/configの作成
    
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
#   kube-apiserver.service
#   kube-controller-manager.service
#   kube-scheduler.service
#   kubelet.service
#   kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=false  --log-dir=/var/log/kubernetes"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=true"

# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://172.16.7.71:8080"

    5、プロファイル/etc/kubernetes/kubeletの作成
    ##
# kubernetes kubelet (minion) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# The port for the info server to serve on
KUBELET_PORT="--port=10250"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=172.16.7.72"

## location of the api-server
KUBELET_API_SERVER="--api-servers=http://172.16.7.71:8080"

# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

# Add your own!
KUBELET_ARGS="--cgroup-driver=systemd --allow-privileged=true"

    6、クbeletを起動する
    # systemctl daemon-reload
# systemctl enable kubelet
# systemctl start kubelet

    7、検証:masterでkubectl get noを起動すれば表示できる
    node->kube-proxy
    1、バイナリファイルを/usr/local/binにコピーする
    2、新規プロファイル/usr/lib/systemd/system/kube-proxy.service 
    [Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=network.target

[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/local/bin/kube-proxy             $KUBE_LOGTOSTDERR             $KUBE_LOG_LEVEL             $KUBE_MASTER             $KUBE_PROXY_ARGS
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

    3、新規プロファイル/etc/kubernetes/proxy
    ###
# kubernetes proxy config

# default config should be adequate

# Add your own!
KUBE_PROXY_ARGS="--master=http://172.16.7.1:8080 --proxy-mode=iptables --logtostderr=false --log-dir=/var/log/kubernetes --alsologtostderr=false --v=2 "

    4、起動
    # systemctl daemon-reload
# systemctl enable kube-proxy
# systemctl start kube-proxy
    HadoopクラスタのHA高可用性アーキテクチャの構築(超詳細ステップ+検証済み)
    EditText cursor位置の設定