docker swarmクラスタおよびマルチホストoverlayネットワークテスト

28899 ワード

dockerのswarmクラスタはすでにマルチホストのoverlayネットワークをサポートしており、現在テストしたところ、インストールと構成が非常に便利で、k 8 sに比べてインストールと構成が簡単であることが分かった.
  • テスト環境は2台の仮想マシンを使用してテストされ、オペレーティングシステムはubuntu 16.04.04、システムがカーネルを持っているのは4.2で、overlayには3.16以上のカーネルバージョンが必要であることに注意してください.

  • ホスト名IPコメント
    ubuntu1	192.168.12.121	manger
    ubuntu2	192.168.12.122	worker
    
  • インストールdockerすべてのホストにdockerをインストールし、公式APTソースを使用します.
  • # docker
    apt-get remove docker docker-engine docker.io
    
    # 
    apt-get install \
        linux-image-extra-$(uname -r) \
        linux-image-extra-virtual
    
    # Docker APT 
    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
    apt-key fingerprint 0EBFCD88
    
    # APT , 
    add-apt-repository \
       "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu/ \
       $(lsb_release -cs) \
       stable"
    
    # docker
    apt-get update
    apt-get install docker-ce
    
    3.  
     ubuntu iptables-persistent 
    
    # 
    apt-get install iptables-persistent
    
    # 
    /etc/init.d/iptables-persistent flush
    
    # manager :
    iptables -A INPUT -p tcp --dport 22 -j ACCEPT
    iptables -A INPUT -p tcp --dport 2376 -j ACCEPT
    iptables -A INPUT -p tcp --dport 2377 -j ACCEPT
    iptables -A INPUT -p tcp --dport 7946 -j ACCEPT
    iptables -A INPUT -p udp --dport 7946 -j ACCEPT
    iptables -A INPUT -p udp --dport 4789 -j ACCEPT
    
    # worker :
    iptables -A INPUT -p tcp --dport 22 -j ACCEPT
    iptables -A INPUT -p tcp --dport 2376 -j ACCEPT
    iptables -A INPUT -p tcp --dport 7946 -j ACCEPT
    iptables -A INPUT -p udp --dport 7946 -j ACCEPT
    iptables -A INPUT -p udp --dport 4789 -j ACCEPT
    
    # 
    /etc/init.d/iptables-persistent save
    
    # docker
    service docker restart
     : 
    – TCP port 2376 for secure Docker client communication. This port is required for Docker Machine to work. Docker Machine is used to orchestrate Docker hosts. 
    – TCP port 2377. This port is used for communication between the nodes of a Docker Swarm or cluster. It only needs to be opened on manager nodes. 
    – TCP and UDP port 7946 for communication among nodes (container network discovery). 
    – UDP port 4789 for overlay network traffic (container ingress networking).
    
  • swarmクラスタを構成してmanagerノード上でクラスタを初期化し、manager上で次のコマンドを実行します:
  • root@ubuntu1:/etc/apt  docker swarm init --advertise-addr 192.168.12.121
    
    Swarm initialized: current node (tg8klhxnuk89tya2lhe35tqx7) is now a manager.
    
    To add a worker to this swarm, run the following command:
    
        docker swarm join --token SWMTKN-1-4m8sl3yl15aop8g7045evqcdh7yxvkrg6be2hhatz2wcyne4d2-ed56hwycegzmq18bvpm3pmodz 10.16.16.56:2377
    
    To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
    

    workerノードをクラスタに追加し、workerで次のコマンドを実行します.
    docker swarm join --token SWMTKN-1-4m8sl3yl15aop8g7045evqcdh7yxvkrg6be2hhatz2wcyne4d2-ed56hwycegzmq18bvpm3pmodz 10.16.16.56:2377
     , manager :
    
    root@ubuntu1:/etc/apt# docker node ls
    ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
    g6f1pd68qjk7wqggtc4zyf *   ubuntu1             Ready               Active              Leader              18.06.0-ce
    mzdlapyot4zb7dj15t0wcy99h     ubuntu2             Ready               Active                                  18.06.0-ce
    
    root@ubuntu1:/etc/apt# 
    root@ubuntu1:/etc/apt# docker network ls
    NETWORK ID          NAME                DRIVER              SCOPE
    b76e6c3cada        bridge              bridge              local
    a02511ca57        docker_gwbridge     bridge              local
    bb0a7a05d2c5        host                host                local
    ojjiuarwgrpm        ingress             overlay             swarm
    dac67e9965        none                null                local
    
    5.  overlay 
    # 
    root@ubuntu1:/etc/apt# docker network create -d overlay --subnet=192.168.0.0/24 --gateway=192.168.0.254 --attachable testnetwork
    crsggk0wycauo9kjwj8z00f1
    
    # 
    root@ubuntu1:/etc/apt# docker network ls
    NETWORK ID          NAME                DRIVER              SCOPE
    b76e6c3cada        bridge              bridge              local
    a02511ca57        docker_gwbridge     bridge              local
    bb0a7a05d2c5        host                host                local
    ojjiuarwgrpm        ingress             overlay             swarm
    dac67e9965        none                null                local
    crsggk0wyca        testnetwork         overlay             swarm
    
    # 
    root@ubuntu1:/etc/apt# docker network inspect testnetwork
    [
        {
            "Name": "testnetwork",
            "Id": "0crsggk0wycauo9kjwj8z00f1",
            "Created": "2018-08-09T17:05:06.757781593Z",
            "Scope": "swarm",
            "Driver": "overlay",
            "EnableIPv6": false,
            "IPAM": {
                "Driver": "default",
                "Options": null,
                "Config": [
                    {
                        "Subnet": "192.168.0.0/24",
                        "Gateway": "192.168.0.254"
                    }
                ]
            },
            "Internal": false,
            "Attachable": true,
            "Ingress": false,
            "ConfigFrom": {
                "Network": ""
            },
            "ConfigOnly": false,
            "Containers": null,
            "Options": {
                "com.docker.network.driver.overlay.vxlanid_list": "4097"
            },
            "Labels": null
        }
    ]
    
  • コンテナネットワークテストmanagerノードにコンテナbusybox 1を作成する:
  • root@ubuntu1:/etc/apt# docker run -itd --name=busybox1 --network=testnetwork busybox /bin/sh
    Unable to find image 'busybox:latest' locally
    latest: Pulling from library/busybox
    c5a7da1afbc: Pull complete 
    Digest: sha256:cb63aa0641a885f54de20f61d152187419e8f6b159ed11a251a09d115fdff9bd
    Status: Downloaded newer image for busybox:latest
    a0c723dd2990813a07d1e9d95b8924edea0bf4e507471ebb619a3ad68ee3a70
    
    # , IP LB IP
    root@ubuntu1:/etc/apt# docker network inspect testnetwork                                   
    [
        {
            "Name": "testnetwork",
            "Id": "0crsggk0wycauo9kjwj8z00f1",
            "Created": "2018-08-10T01:06:34.830676798+08:00",
            "Scope": "swarm",
            "Driver": "overlay",
            "EnableIPv6": false,
            "IPAM": {
                "Driver": "default",
                "Options": null,
                "Config": [
                    {
                        "Subnet": "192.168.0.0/24",
                        "Gateway": "192.168.0.254"
                    }
                ]
            },
            "Internal": false,
            "Attachable": true,
            "Ingress": false,
            "ConfigFrom": {
                "Network": ""
            },
            "ConfigOnly": false,
            "Containers": {
                "1a0c723dd2990813a07d1e9d95b8924edea0bf4e507471ebb619a3ad68ee3a70": {
                    "Name": "busybox1",
                    "EndpointID": "95bc3d1c0ddd3aacb15070aafb8aebb9dc31029ca0684288af72081ab34ad085",
                    "MacAddress": "02:42:c0:a8:00:03",
                    "IPv4Address": "192.168.0.3/24",
                    "IPv6Address": ""
                },
                "lb-testnetwork": {
                    "Name": "testnetwork-endpoint",
                    "EndpointID": "0f28c078488afb7a19a2d8ec37bb6df5991f763db2f714c0f8c1f23728fb5b46",
                    "MacAddress": "02:42:c0:a8:00:01",
                    "IPv4Address": "192.168.0.1/24",
                    "IPv6Address": ""
                }
            },
            "Options": {
                "com.docker.network.driver.overlay.vxlanid_list": "4097"
            },
            "Labels": {},
            "Peers": [
                {
                    "Name": "67603512578a",
                    "IP": "192.168.12.121"
                },
                {
                    "Name": "f77f0897a85b",
                    "IP": "192.168.12.122"
                }
            ]
        }
    ]
     worker busybox2:
    
    root@ubuntu2:~# docker run -itd --name=busybox2 --network=testnetwork busybox /bin/sh
    Unable to find image 'busybox:latest' locally
    latest: Pulling from library/busybox
    c5a7da1afbc: Pull complete 
    Digest: sha256:cb63aa0641a885f54de20f61d152187419e8f6b159ed11a251a09d115fdff9bd
    Status: Downloaded newer image for busybox:latest
    bb544eaf149086f93e6c35d9098a937282ed442be582e4516c24ac5fce9100da
    
    root@ubuntu2:~# docker network inspect testnetwork
    [
        {
            "Name": "testnetwork",
            "Id": "0crsggk0wycauo9kjwj8z00f1",
            "Created": "2018-08-10T01:07:23.121549142+08:00",
            "Scope": "swarm",
            "Driver": "overlay",
            "EnableIPv6": false,
            "IPAM": {
                "Driver": "default",
                "Options": null,
                "Config": [
                    {
                        "Subnet": "192.168.0.0/24",
                        "Gateway": "192.168.0.254"
                    }
                ]
            },
            "Internal": false,
            "Attachable": true,
            "Ingress": false,
            "ConfigFrom": {
                "Network": ""
            },
            "ConfigOnly": false,
            "Containers": {
                "bb544eaf149086f93e6c35d9098a937282ed442be582e4516c24ac5fce9100da": {
                    "Name": "busybox2",
                    "EndpointID": "bd61b48e4b066d9e9ca81a267ee0c554c047f6bd68e16795fff81180f4b3fcdd",
                    "MacAddress": "02:42:c0:a8:00:04",
                    "IPv4Address": "192.168.0.4/24",
                    "IPv6Address": ""
                },
                "lb-testnetwork": {
                    "Name": "testnetwork-endpoint",
                    "EndpointID": "f912103a56c705c31c8ee9476dfca1c53d0e2e321781a38338b3a180c3d08f36",
                    "MacAddress": "02:42:c0:a8:00:02",
                    "IPv4Address": "192.168.0.2/24",
                    "IPv6Address": ""
                }
            },
            "Options": {
                "com.docker.network.driver.overlay.vxlanid_list": "4097"
            },
            "Labels": {},
            "Peers": [
                {
                    "Name": "67603512578a",
                    "IP": "192.168.12.121"
                },
                {
                    "Name": "f77f0897a85b",
                    "IP": "192.168.12.122"
                }
            ]
        }
    ]
    

    作成したばかりの2つのコンテナ内で互いにPINGをテストします.
    #manager 
    root@ubuntu1:/etc/apt# docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED              STATUS              PORTS               NAMES
    a0c723dd299        busybox             "/bin/sh"           About a minute ago   Up About a minute                       busybox1
    
    root@ubuntu1:/etc/apt# docker exec -it 1a0c723dd299 ping 192.168.0.4
    PING 192.168.0.4 (192.168.0.4): 56 data bytes
     bytes from 192.168.0.4: seq=0 ttl=64 time=1.040 ms
     bytes from 192.168.0.4: seq=1 ttl=64 time=0.763 ms
     bytes from 192.168.0.4: seq=2 ttl=64 time=0.854 ms
     bytes from 192.168.0.4: seq=3 ttl=64 time=0.745 ms
     bytes from 192.168.0.4: seq=4 ttl=64 time=0.846 ms
     bytes from 192.168.0.4: seq=5 ttl=64 time=0.716 ms
     bytes from 192.168.0.4: seq=6 ttl=64 time=0.889 ms
    ^C
    --- 192.168.0.4 ping statistics ---
     packets transmitted, 7 packets received, 0% packet loss
    round-trip min/avg/max = 0.716/0.836/1.040 ms
    
    #worker 
    root@ubuntu2:~# docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
    bb544eaf1490        busybox             "/bin/sh"           2 minutes ago       Up 2 minutes                            busybox2
    root@ubuntu2:~# docker exec -it bb544eaf1490 ping 192.168.0.3
    PING 192.168.0.3 (192.168.0.3): 56 data bytes
     bytes from 192.168.0.3: seq=0 ttl=64 time=0.754 ms
     bytes from 192.168.0.3: seq=1 ttl=64 time=0.677 ms
     bytes from 192.168.0.3: seq=2 ttl=64 time=0.873 ms
    ^C
    --- 192.168.0.3 ping statistics ---
     packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 0.677/0.768/0.873 ms
    root@ubuntu2:~# 
     2 ping 。
    
  • クラウド環境制限dockerオリジナルoverlayネットワークで使用されるのは標準的なvxlanプロトコルであり、使用されるポートも標準的なvxlanポートである(UDP 4789).各クラウド環境、例えばアリクラウド、テンセントクラウドもvxlanを使用しています.だから衝突があって、UDP 4789ネットワークは通じません.今のところ融通のきく方法は見つからない.dockerはこれまでカスタムvxlanポートをサポートしていません.(テンセント雲の黒石環境で検証したが、確定できない)
  • 参照https://docs.docker.com/network/network-tutorial-overlay/

  • https://www.digitalocean.com/community/tutorials/how-to-configure-the-linux-firewall-for-docker-swarm-on-ubuntu-16-04