ElasticSearch+SearchGuardクラスタ構築

16848 ワード
背景紹介
業務元のESクラスタは証明書の期限切れのため、証明書交換操作を行う必要がある.証明書の更新後にクラスタ全体を再起動する必要があり、ビジネス感度が高いため、クラスタ全体の対外サービス停止は許容できません.
ソリューション
  • OPは新しいESクラスタの配置を担当し、バージョン構成は元のクラスタと一致する
    • である.
  • RD開発プログラム担当
  • 履歴データを新しいクラスタ
    • にインポートする.
  • 業務サービスは2セットのESクラスタ機能
    • を実現する.
  • の2セットのクラスタデータが一致した後、OPとRDは業務流量
    • の切り替えに協力した.
  • OP既存クラスタ更新証明書
    • ElasticSearchクラスタ
    の準備を
  • ESバージョン5.5.1
  • Search Guard
  • JAVAバージョンjdk 1.8.0_161

    • クラスタ構成
    3  Master Node (56 /128G/3.7T    )
10.90.104.133
10.90.105.133
10.90.106.133

5  Data Node (56 /128G/3.7T    )
10.90.107.132
10.90.108.132
10.90.109.133
10.90.110.133
10.90.111.133

  : 9201/9301

    システムちょうせい
        
$ sudo sysctl -w vm.max_map_count=262144

    
$ grep vm.max_map_count /etc/sysctl.conf
$ echo vm.max_map_count=262144 >> /etc/sysctl.conf

or 
$ vi /etc/sysctl.conf
vm.max_map_count=262144

    取付JAVA
       jdk1.8.0_161(   404，        )
$ wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u161-b12/2f38c3b165be4555a1fa6e98c45e0808/jdk-8u161-linux-x64.tar.gz

    ,     
$ mkdir -p /usr/local/java/jdk1.8.0_161/
$ cp jdk-8u161-linux-x64.tar.gz /usr/local/java/jdk1.8.0_161/ 
$ cd /usr/local/java/jdk1.8.0_161/ 
$ tar zxf jdk-8u161-linux-x64.tar.gz && rm -rf jdk-8u161-linux-x64.tar.gz

   java，   alternatives       java    (command --install    )
$ alternatives --install /usr/bin/java java $JAVA_18_161/bin/java 2
$ alternatives --config java

There are 4 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
   1           /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64/jre/bin/java
*+ 2           /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.65-3.b17.el7.x86_64/jre/bin/java
   3           /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java
   4           /usr/local/java/jdk1.8.0_161/bin/java

Enter to keep the current selection[+], or type selection number: 4

   jar , javac 
alternatives --install /usr/bin/jar jar /usr/local/java/jdk1.8.0_161/bin/jar 2
alternatives --install /usr/bin/javac javac /usr/local/java/jdk1.8.0_161/bin/javac 2
alternatives --set jar /usr/local/java/jdk1.8.0_161/bin/jar
alternatives --set javac /usr/local/java/jdk1.8.0_161/bin/javac

      
$ vim /etc/bashrc
export JAVA_HOME=/usr/local/java/jdk1.8.0_161
export JRE_HOME=/usr/local/java/jdk1.8.0_161/jre
export PATH=/root/perl5/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/usr/local/java/jdk1.8.0_161/bin:/usr/local/java/jdk1.8.0_161/jre/bin

  java  
$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

    取付ElasticSearch
        
$ adduser -U -s /sbin/nologin elasticsearch

    
$ mkdir -p /usr/local/elasticsearch
$ mkdir -p /data/elasticsearch/my-project
$ mkdir -p /data/logs/elasticsearch/my-project/{log,pid}

  es 5.5.1
$ cd /usr/local/elasticsearch
$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.1.tar.gz

    
$ tar zxf elasticsearch-5.5.1.tar.gz && rm -rf elasticsearch-5.5.1.tar.gz
$ mv elasticsearch-5.5.1 my-project-es
    Search Guardプラグインのインストール
        
$ cd {PROJECT_NAME}
$ wget http://search.maven.org/remotecontent?filepath=com/floragunn/search-guard-5/5.5.1-15/search-guard-5-5.5.1-15.zip

    
$ bin/elasticsearch-plugin install search-guard-5-5.5.1-15.zip

    証明書の生成
        (        )
$ git clone https://github.com/floragunncom/search-guard-ssl.git

    
$ cd search-guard-ssl/example-pki-scripts/
$ ./clean.sh
$ ./gen_root_ca.sh kubernetes- kubernetes-                      # COMMAND  
$ ./gen_node_cert.sh k8s-es-node kubernetes- kubernetes-        # COMMAND   
$ ./gen_client_node_cert.sh sgadmin kubernetes- kubernetes-     # COMMAND   

      config    
$ cp *jks /usr/local/elasticsearch/my-project/config/

    構成ElasticSearch
        
$ cd /usr/local/elasticsearch/my-project/config/
$ mv elasticsearch.yml{,bak}

Node Master (  master     ):
   :     ，    http
$ vim elasticsearch.yml

cluster.name: my-project-es

node.name: 10.90.104.133
node.master: true
node.data: false
node.ingest: false

path:
    data:
    - /data/elasticsearch/my-project
    logs: /data/logs/elasticsearch/my-project/log

thread_pool.index.queue_size: 1000
thread_pool.bulk.queue_size: 1000

bootstrap.memory_lock: true

network.host: 10.90.104.133
http.port: 9201
transport.tcp.port: 9301

http:
    enabled: false
    compression: true
    cors:
        enabled: true
        allow-origin: "*"
        allow-headers: Authorization

discovery.zen.ping.unicast.hosts: [10.90.104.133:9301,10.90.105.133:9301,10.90.106.133:9301]
discovery.zen.minimum_master_nodes: 2

searchguard.ssl.transport.keystore_filepath: node-k8s-es-node-keystore.jks
searchguard.ssl.transport.keystore_password: kubernetes-
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: kubernetes-
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.authcz.admin_dn:
- CN=sgadmin,OU=client,O=client,L=test,C=DE

    :
$ vim jvm.options
...
Xms20g
Xmx20g
...


Node Data ( 5          ):
   :         http   
$ vim elasticsearch.yml

cluster.name: my-project-es

node.name: 10.90.107.132
node.master: false
node.data: true
node.ingest: true

path:
    data:
    - /data/elasticsearch/my-project
    logs: /data/logs/elasticsearch/my-project/log

thread_pool.index.queue_size: 1000
thread_pool.bulk.queue_size: 1000

bootstrap.memory_lock: true

network.host: 10.90.107.132
http.port: 9201
transport.tcp.port: 9301

http:
    enabled: true
    compression: true
    cors:
        enabled: true
        allow-origin: "*"
        allow-headers: Authorization

discovery.zen.ping.unicast.hosts: [10.90.104.133:9301,10.90.105.133:9301,10.90.106.133:9301]
discovery.zen.minimum_master_nodes: 2

searchguard.ssl.transport.keystore_filepath: node-k8s-es-node-keystore.jks
searchguard.ssl.transport.keystore_password: kubernetes-
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: kubernetes-
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.authcz.admin_dn:
- CN=sgadmin,OU=client,O=client,L=test,C=DE

    :
$ vim jvm.options
...
Xms63g
Xmx63g
...

    ユーザー権限およびパスワード
    $ cd /usr/local/elasticsearch/my-project/plugins/search-guard-5

    
$ sh tools/hash.sh -p ICdsy1B3j68Tr4w2
$2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe

        ,      myuser
$ vim sgconfig/sg_internal_users.yml

# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
admin:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  #password is: ICdsy1B3j68Tr4w2
logstash:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  #password is: ICdsy1B3j68Tr4w2
kibanaserver:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  #password is: ICdsy1B3j68Tr4w2
kibanaro:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  #password is: ICdsy1B3j68Tr4w2
  roles:
    - kibanarole
readall:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  #password is: ICdsy1B3j68Tr4w2
myuser:
  hash: $2a$12$BrsaHcr6q0SawxbJtjN46ORVcYsdLH1hQSwVTW4vbUAOwUckpWCbe
  # password is: ICdsy1B3j68Tr4w2
 
        (  admin  )
$ vim sgconfig/sg_roles_mapping.yml

...

#             ，        
sg_all_access:
  users:
    - admin
    - myuser
    
...

    サービス管理スクリプトの追加
    $ vim /usr/lib/systemd/system/es-my-project.service

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Environment=ES_HOME=/usr/local/elasticsearch/my-project
Environment=CONF_DIR=/usr/local/elasticsearch/my-project/config
Environment=PID_DIR=/data/logs/elasticsearch/my-project/pid
WorkingDirectory=/usr/local/elasticsearch/my-project

User=elasticsearch
Group=elasticsearch

ExecStartPre=/usr/local/elasticsearch/my-project/bin/elasticsearch-systemd-pre-exec

ExecStart=/usr/local/elasticsearch/my-project/bin/elasticsearch \
        -p \${PID_DIR}/elasticsearch.pid \
        -Edefault.path.conf=\${CONF_DIR}

StandardOutput=journal
StandardError=inherit
LimitNOFILE=65536
LimitNPROC=2048
LimitMEMLOCK=infinity
TimeoutStopSec=0
KillSignal=SIGTERM
KillMode=process
SendSIGKILL=no
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

    ユーザーの変更
    chown -R elasticsearch:elasticsearch /usr/local/elasticsearch
chown -R elasticsearch:elasticsearch /data/logs/elasticsearch
chown -R elasticsearch:elasticsearch /data/elasticsearch
chown -R elasticsearch:elasticsearch /usr/lib/systemd/system/es-my-project.service

    サービスの開始
          
sudo systemctl daemon-reload
sudo systemctl enable es-my-project.service
sudo systemctl start es-my-project

    sguard初期化パスワード
    sguard    ，                (      master  )
  sgconfig          ，         ES  。

$ cd /usr/local/elasticsearch/my-project/plugins/search-guard-5/tools
$ ./sgadmin.sh --hostname 10.90.104.133 --port 9301 \
    -cd ../sgconfig/ \
    -ks /usr/local/elasticsearch/my-project/config/sgadmin-keystore.jks \
    -kspass kubernetes- \
    -ts ./usr/local/elasticsearch/my-project/config/truststore.jks  \
    -tspass kubernetes- \
    -nhnv --diagnose -cn my-project-es

    クラスタのステータスの表示
    $ curl -u myuser:ICdsy1B3j68Tr4w2 -XGET http://10.90.107.132:9201/_cluster/health?pretty

{
  "cluster_name" : "my-project-es",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 3,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 4,
  "active_shards" : 9,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

    Kibana配備
    Dockerfile 
    FROM docker.xxx.com/library/kibana-searchguard:5.5.1

ENV CLUSTER_NAME=my-project-es \
    SERVER_BASEPATH= \
    ELASTICSEARCH_URL="http://10.90.107.132:9201" \
    XPACK_SECURITY_ENABLED=false \
    XPACK_GRAPH_ENABLED=false \
    XPACK_ML_ENABLED=false \
    ELASTICSEARCH_USERNAME=kibanaserver \
    ELASTICSEARCH_PASSWORD=ICdsy1B3j68Tr4w2

# port is 5601

    内部ウェアハウス・ミラー・ベースの構築
    ミラーの構築+コンテナの生成
    $ docker build -t my-project-kibana .
$ docker run -d my-project-kibana

    kibanaへのログイン
    初期化の設定
    証明書の交換
    新しく生成された証明書ファイル*.jksは、元のクラスタconfigの下にコピーされ、ESクラスタとsearch guardを再起動すればよい.
    転載先:https://juejin.im/post/5d4d37e8e51d453b7779d504
    2つの従来の動的計画アルゴリズムの問題,2つの邪門の解
    k 8 sクラスタ引き抜きミラーエラー:not exist or may require docker login