Redisクラスタモードの分散クラスタモード

34585 ワード
14. ビッグデータ ---14.4. Redis レンガを運ぶ道具のRedis

前言


Redisクラスタモードは主に2種類あります.
  • マスタスレーブクラスタ
  • 分散クラスタ.

    • 前者は主に高可用性または読み書き分離のためであり,後者はデータのより良い記憶のために負荷が均衡している.本文は主に主従クラスタについて説明する.本章では,後半部分,Redisクラスタについて述べる.
    本明細書に関連するコードおよびプロファイルはgithubにアップロードされました.アドレス:https://github.com/SeanYanxml/bigdata

    げんり


    Redisは負荷等化を実現するためにクラスタモードを提供する.3つのノードを例にとると、クラスタモードは、1〜1500枚を1〜5000枚、5000〜1000、10000〜15000に分割することに相当する.各ノードはデータ・スライスを分割します.これにより、1つのノードがダウンタイムした後、このノードがバックアップされていない場合、このセグメントは使用できなくなります.したがって、クラスタ内の各ノードには、冷却としてスレーブノードが配備されることが推奨されています.配置原理図を以下に示す(暫定).

    配置


    それほど多くの機器がないため、6つのノード(3主3従)、つまり公式サイトが推奨するモードを単機で配置するのが一般的です.
    主な手順は次のとおりです.
  • は、クラスタのコードが割り当てられたときにrubyスクリプトがインストールされるため、rubyをインストールします.
  • 分配クラスタ
  • クラスタ結果を表示

    • 配備コード

    # mkdir cluster-test
# cd cluster-test
# mkdir 7000 7001 7002 7003 7004 7005
#           redis.conf         
#   6   
# nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
＃   ruby        
./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
    # redis.conf
port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
    ###   redis  7000 7001 7002 7003 7004 7005

touch start-all.sh
###      
cd 7000
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7001
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7002
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7003
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7004
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &
cd ../7005
nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

chmod +x start-all.sh
./start-all.sh

ps -ef|grep redis     
[cpic@cpic-redis2-77 src]$ ps -ef|grep redis
cpic     19876     1  0 3 21 ?       05:14:19 ./redis-server *:6379
cpic     19886     1  0 3 21 ?       06:17:36 ./redis-sentinel *:26379 [sentinel]
cpic     25971     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7000 [cluster]
cpic     25972     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7001 [cluster]
cpic     25973     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7002 [cluster]
cpic     25974     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7003 [cluster]
cpic     25975     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7004 [cluster]
cpic     25976     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7005 [cluster]
cpic     26273 24815  0 15:15 pts/0    00:00:00 grep --color=auto redis

    成功の結果

    ＃ create         
[root@cpic-redis-76 src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003
   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96
S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004
   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3
S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005
   replicates 6522a9ddec5b40cade141793351df44fe40a529f
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3
M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 6522a9ddec5b40cade141793351df44fe40a529f
S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003
   slots: (0 slots) slave
   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
    #      node.conf  
[root@cpic-redis-76 7005]# cat nodes.conf
1c7cbff1069612078d63329e5841c7430757a2cd 127.0.0.1:7003@17003 slave f5cc2de688640de572d16d72c2f174de027609a6 0 1516672833000 4 connected
dd09a06827e9bfeab3876f4464cd467f6d1279f0 127.0.0.1:7002@17002 master - 0 1516672833086 3 connected 10923-16383
5863f9ee11d3ee3beef466d76f48f221b48aa6b5 127.0.0.1:7004@17004 slave 3ed6531ec985e50e87ab8436d49f4c32ce82bdef 0 1516672832000 5 connected
3ed6531ec985e50e87ab8436d49f4c32ce82bdef 127.0.0.1:7001@17001 master - 0 1516672832000 2 connected 5461-10922
9013209020c2fe2322df8c420b1c251fe9c84955 127.0.0.1:7005@17005 myself,slave dd09a06827e9bfeab3876f4464cd467f6d1279f0 0 1516672831000 6 connected
f5cc2de688640de572d16d72c2f174de027609a6 127.0.0.1:7000@17000 master - 0 1516672832885 1 connected 0-5460
vars currentEpoch 6 lastVoteEpoch 0
    #    log  
[root@cpic-redis-76 7005]# cat start.log
nohup:     
5627:C 22 Jan 21:02:58.212 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
5627:C 22 Jan 21:02:58.212 # Redis version=4.0.6, bits=64, commit=00000000, modified=0, pid=5627, just started
5627:C 22 Jan 21:02:58.212 # Configuration loaded
5627:M 22 Jan 21:02:58.214 * Increased maximum number of open files to 10032 (it was originally set to 1024).
5627:M 22 Jan 21:02:58.215 * Node configuration loaded, I'm 9013209020c2fe2322df8c420b1c251fe9c84955
5627:M 22 Jan 21:02:58.216 * Running mode=cluster, port=7005.
5627:M 22 Jan 21:02:58.216 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
5627:M 22 Jan 21:02:58.216 # Server initialized
5627:M 22 Jan 21:02:58.216 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
5627:M 22 Jan 21:02:58.216 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
5627:M 22 Jan 21:02:58.216 * Ready to accept connections
5627:M 23 Jan 10:00:27.212 # configEpoch set to 6 via CLUSTER SET-CONFIG-EPOCH
5627:M 23 Jan 10:00:27.286 # IP address for this node updated to 127.0.0.1
5627:S 23 Jan 10:00:31.238 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
5627:S 23 Jan 10:00:31.239 # Cluster state changed: ok
5627:S 23 Jan 10:00:31.981 * Connecting to MASTER 127.0.0.1:7002
5627:S 23 Jan 10:00:31.981 * MASTER  SLAVE sync started
5627:S 23 Jan 10:00:31.981 * Non blocking connect for SYNC fired the event.
5627:S 23 Jan 10:00:31.981 * Master replied to PING, replication can continue...
5627:S 23 Jan 10:00:31.982 * Trying a partial resynchronization (request 033127a1144abc3683a83a6b7257c2b56d2bd0b8:1).
5627:S 23 Jan 10:00:31.983 * Full resync from master: e5e51569d1f49762952e6cc05e1f13ddff68719e:0
5627:S 23 Jan 10:00:31.983 * Discarding previously cached master state.
5627:S 23 Jan 10:00:32.084 * MASTER  SLAVE sync: receiving 175 bytes from master
5627:S 23 Jan 10:00:32.084 * MASTER  SLAVE sync: Flushing old data
5627:S 23 Jan 10:00:32.084 * MASTER  SLAVE sync: Loading DB in memory
5627:S 23 Jan 10:00:32.084 * MASTER  SLAVE sync: Finished with success
5627:S 23 Jan 10:00:32.085 * Background append only file rewriting started by pid 10164
5627:S 23 Jan 10:00:32.108 * AOF rewrite child asks to stop sending diffs.
10164:C 23 Jan 10:00:32.108 * Parent agreed to stop sending diffs. Finalizing AOF...
10164:C 23 Jan 10:00:32.108 * Concatenating 0.00 MB of AOF diff received from parent.
10164:C 23 Jan 10:00:32.108 * SYNC append only file rewrite performed
10164:C 23 Jan 10:00:32.109 * AOF rewrite: 0 MB of memory used by copy-on-write
5627:S 23 Jan 10:00:32.181 * Background AOF rewrite terminated with success
5627:S 23 Jan 10:00:32.181 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
5627:S 23 Jan 10:00:32.181 * Background AOF rewrite finished successfully

    BUG

  • BUG1 
    • [root@cpic-redis-76 src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
[ERR] Node 127.0.0.1:7005 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
    :       dump.rdb/appendonly.aof/nodes.conf  。
  • BUG2 
    • ruby       。

    詳細については、Redis Cluster環境インストールレコードを参照してください.

    Reference


    Redis Clusterメンテナンス環境インストール記録を参照
    スライドの有効期限を1秒に設定するとどうなりますか
    Redis使用シーン:認証コードの送信回数を制限する