我们在18台主机上安装了redis集群，有6个主站。通过过去几周对redis服务的监控。我发现它经常无缘无故的关闭。Redis服务在docker中抵抗，我们用"--restart unless-stopped "配置docker。 虽然这有助于在关机后重启redis，但我不知道为什么redis服务会经常关机。我把redis的日志贴在这里。在不到20分钟的时间里，它关闭了3次。但我们从来没有发送任何关闭命令或信号给它。所以，如果有人能帮忙，我真的很感激。
1:S 20 May 2020 05:04:48.065 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 4cb5a97a22bfb9bcae0d9faa009c49e53e6ffbfe
1:S 20 May 2020 05:04:48.066 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 04b7946cc987adfeb89ee3d86f8af0a4b9155af6
1:S 20 May 2020 05:04:48.066 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 864d70c9efbc5fa13311ac4e09973d9731baee1d
1:S 20 May 2020 05:04:48.066 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 34f172f159981388f753aa3db2f7818ef5b75324
1:S 20 May 2020 05:04:48.066 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 4a2b535565d7c82268587f3ce80e4e038ec99c27
1:S 20 May 2020 05:04:48.066 # Cluster state changed: fail
1:S 20 May 2020 05:04:48.534 * FAIL message received from 3f48fef6a5294bc3fe937fb2fb1310d72defb047 about 18dacdd41de04f67ea29bd5a0608698af9bbc1ce
1:S 20 May 2020 05:04:48.951 * Clear FAIL state for node 4a2b535565d7c82268587f3ce80e4e038ec99c27: replica is reachable again.
1:S 20 May 2020 05:04:49.161 * Clear FAIL state for node 04b7946cc987adfeb89ee3d86f8af0a4b9155af6: replica is reachable again.
1:S 20 May 2020 05:04:49.369 * Clear FAIL state for node 34f172f159981388f753aa3db2f7818ef5b75324: replica is reachable again.
1:S 20 May 2020 05:04:49.431 # Cluster state changed: ok
1:S 20 May 2020 05:04:49.577 * Clear FAIL state for node 4cb5a97a22bfb9bcae0d9faa009c49e53e6ffbfe: replica is reachable again.
1:S 20 May 2020 05:04:49.583 * Clear FAIL state for node 864d70c9efbc5fa13311ac4e09973d9731baee1d: master without slots is reachable again.
1:S 20 May 2020 05:04:50.197 * Clear FAIL state for node 18dacdd41de04f67ea29bd5a0608698af9bbc1ce: master without slots is reachable again.
1:S 20 May 2020 05:05:18.220 # User requested shutdown...
1:S 20 May 2020 05:05:18.220 * Calling fsync() on the AOF file.
1:S 20 May 2020 05:05:18.221 * Saving the final RDB snapshot before exiting.
1:S 20 May 2020 05:05:18.225 * DB saved on disk
1:S 20 May 2020 05:05:18.225 # Redis is now ready to exit, bye bye...
1:C 20 May 2020 05:05:27.512 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 20 May 2020 05:05:27.512 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 20 May 2020 05:05:27.512 # Configuration loaded
1:M 20 May 2020 05:05:27.551 * Node configuration loaded, I'm a90232a10cd0b4a49d10e2f829bf46e38687e266
1:M 20 May 2020 05:05:27.553 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 20 May 2020 05:05:27.553 # Server initialized
1:M 20 May 2020 05:05:27.553 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 20 May 2020 05:05:27.561 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 20 May 2020 05:05:27.564 * DB loaded from append only file: 0.002 seconds
1:M 20 May 2020 05:05:27.564 * Ready to accept connections
1:S 20 May 2020 05:05:27.573 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
1:S 20 May 2020 05:05:27.574 # Cluster state changed: ok
1:S 20 May 2020 05:05:28.906 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:05:28.948 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:05:28.949 # Error condition on socket for SYNC: Connection refused
1:S 20 May 2020 05:05:30.240 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:05:30.278 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:05:30.280 # Error condition on socket for SYNC: Connection refused
1:S 20 May 2020 05:05:31.631 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:05:31.653 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:05:31.656 # Error condition on socket for SYNC: Connection refused
1:S 20 May 2020 05:05:32.427 * FAIL message received from 3f48fef6a5294bc3fe937fb2fb1310d72defb047 about 2b29063097f7a75c13389d001f81c8ed8bbd49b9
1:S 20 May 2020 05:05:32.428 # Cluster state changed: fail
1:S 20 May 2020 05:05:32.438 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about ceb1b230cc4a3944583f57299da88500d978cdf7
1:S 20 May 2020 05:05:33.007 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:05:33.049 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:05:33.051 # Error condition on socket for SYNC: Connection refused
1:S 20 May 2020 05:05:34.492 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:05:34.529 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:05:34.531 * Non blocking connect for SYNC fired the event.
1:S 20 May 2020 05:05:34.531 * Master replied to PING, replication can continue...
1:S 20 May 2020 05:05:34.531 * Non blocking connect for SYNC fired the event.
1:S 20 May 2020 05:05:34.531 * Master replied to PING, replication can continue...
1:S 20 May 2020 05:05:34.533 * Trying a partial resynchronization (request 19a0e30f479919de16693aa57dfe522a674d7f50:1).
1:S 20 May 2020 05:05:34.535 * Full resync from master: b480823f3ac5dc08397e06f6a41a4c63f855c4f4:0
1:S 20 May 2020 05:05:34.535 * Discarding previously cached master state.
1:S 20 May 2020 05:05:34.623 * MASTER <-> REPLICA sync: receiving 175 bytes from master
1:S 20 May 2020 05:05:34.623 * MASTER <-> REPLICA sync: Flushing old data
1:S 20 May 2020 05:05:34.624 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 20 May 2020 05:05:34.624 * MASTER <-> REPLICA sync: Finished with success
1:S 20 May 2020 05:05:34.625 * Background append only file rewriting started by pid 18
1:S 20 May 2020 05:05:34.659 # Start of election delayed for 1610 milliseconds (rank #1, offset 0).
1:S 20 May 2020 05:05:34.659 * AOF rewrite child asks to stop sending diffs.
18:C 20 May 2020 05:05:34.659 * Parent agreed to stop sending diffs. Finalizing AOF...
18:C 20 May 2020 05:05:34.659 * Concatenating 0.00 MB of AOF diff received from parent.
18:C 20 May 2020 05:05:34.659 * SYNC append only file rewrite performed
18:C 20 May 2020 05:05:34.660 * AOF rewrite: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:05:34.760 * Background AOF rewrite terminated with success
1:S 20 May 2020 05:05:34.760 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
1:S 20 May 2020 05:05:34.760 * Background AOF rewrite finished successfully
1:S 20 May 2020 05:05:36.331 # Starting a failover election for epoch 27.
1:S 20 May 2020 05:05:37.662 * FAIL message received from 2b29063097f7a75c13389d001f81c8ed8bbd49b9 about 065f3367e205790e0b753936793262d4e6dbb7b3
1:S 20 May 2020 05:05:37.751 * FAIL message received from 2b29063097f7a75c13389d001f81c8ed8bbd49b9 about f9837a4dbc5bfe1fe809a9c5619253feabff6ac7
1:S 20 May 2020 05:05:39.258 * Clear FAIL state for node 065f3367e205790e0b753936793262d4e6dbb7b3: replica is reachable again.
1:S 20 May 2020 05:05:40.208 * FAIL message received from 2b29063097f7a75c13389d001f81c8ed8bbd49b9 about 2e4d17182e38b077e0e4c386ae89084879146434
1:S 20 May 2020 05:05:42.540 # Currently unable to failover: Waiting for votes, but majority still not reached.
1:S 20 May 2020 05:05:42.573 * Clear FAIL state for node f9837a4dbc5bfe1fe809a9c5619253feabff6ac7: master without slots is reachable again.
1:S 20 May 2020 05:05:42.803 * Clear FAIL state for node 2b29063097f7a75c13389d001f81c8ed8bbd49b9: is reachable again and nobody is serving its slots after some time.
1:S 20 May 2020 05:05:43.474 * Clear FAIL state for node ceb1b230cc4a3944583f57299da88500d978cdf7: is reachable again and nobody is serving its slots after some time.
1:S 20 May 2020 05:05:43.474 # Cluster state changed: ok
1:S 20 May 2020 05:05:44.622 * FAIL message received from 3f48fef6a5294bc3fe937fb2fb1310d72defb047 about 35e92062e0173db1480ef53440034b21cad1315b
1:S 20 May 2020 05:05:45.988 * Clear FAIL state for node 2e4d17182e38b077e0e4c386ae89084879146434: replica is reachable again.
1:S 20 May 2020 05:05:48.350 * Clear FAIL state for node 35e92062e0173db1480ef53440034b21cad1315b: replica is reachable again.
1:S 20 May 2020 05:06:06.590 * FAIL message received from 3f48fef6a5294bc3fe937fb2fb1310d72defb047 about 2b29063097f7a75c13389d001f81c8ed8bbd49b9
1:S 20 May 2020 05:06:06.590 # Cluster state changed: fail
1:S 20 May 2020 05:06:07.530 # Cluster state changed: ok
1:S 20 May 2020 05:06:09.103 * FAIL message received from 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2 about 6502f1b3f1bcc7f275672223973abd8a850c6994
1:S 20 May 2020 05:06:09.840 * Clear FAIL state for node 2b29063097f7a75c13389d001f81c8ed8bbd49b9: master without slots is reachable again.
1:S 20 May 2020 05:06:14.185 * Clear FAIL state for node 6502f1b3f1bcc7f275672223973abd8a850c6994: replica is reachable again.
1:S 20 May 2020 05:06:14.583 * FAIL message received from 3f48fef6a5294bc3fe937fb2fb1310d72defb047 about b13db1b013e45f0693bf3447a2313542d775692b
1:S 20 May 2020 05:06:14.583 # Cluster state changed: fail
1:S 20 May 2020 05:06:14.747 # Cluster state changed: ok
1:S 20 May 2020 05:06:18.405 * Clear FAIL state for node b13db1b013e45f0693bf3447a2313542d775692b: master without slots is reachable again.
1:S 20 May 2020 05:06:25.357 * FAIL message received from 04b7946cc987adfeb89ee3d86f8af0a4b9155af6 about 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2
1:S 20 May 2020 05:06:25.357 # Cluster state changed: fail
1:S 20 May 2020 05:06:25.965 # Cluster state changed: ok
1:S 20 May 2020 05:06:28.026 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:06:28.027 * Background saving started by pid 19
19:C 20 May 2020 05:06:28.032 * DB saved on disk
19:C 20 May 2020 05:06:28.033 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:06:28.163 * Background saving terminated with success
1:S 20 May 2020 05:06:29.166 * Clear FAIL state for node 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2: master without slots is reachable again.
1:S 20 May 2020 05:08:52.793 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:06:29.166 * Clear FAIL state for node 6fa6f2946ecc2ca7e7f21f059c217097cb6b27c2: master without slots is reachable again.
1:S 20 May 2020 05:08:52.793 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:08:52.794 * Background saving started by pid 20
20:C 20 May 2020 05:08:52.799 * DB saved on disk
20:C 20 May 2020 05:08:52.799 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:08:52.897 * Background saving terminated with success
1:S 20 May 2020 05:10:10.852 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:10:10.853 * Background saving started by pid 21
21:C 20 May 2020 05:10:10.858 * DB saved on disk
21:C 20 May 2020 05:10:10.859 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:10:10.954 * Background saving terminated with success
1:S 20 May 2020 05:11:44.732 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:11:44.733 * Background saving started by pid 22
22:C 20 May 2020 05:11:44.738 * DB saved on disk
22:C 20 May 2020 05:11:44.739 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:11:44.833 * Background saving terminated with success
1:S 20 May 2020 05:13:06.512 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:13:06.513 * Background saving started by pid 23
23:C 20 May 2020 05:13:06.518 * DB saved on disk
23:C 20 May 2020 05:13:06.518 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:13:06.614 * Background saving terminated with success
1:S 20 May 2020 05:14:10.289 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:14:10.290 * Background saving started by pid 24
24:C 20 May 2020 05:14:10.295 * DB saved on disk
24:C 20 May 2020 05:14:10.296 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:14:10.390 * Background saving terminated with success
1:S 20 May 2020 05:15:11.075 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:15:11.076 * Background saving started by pid 25
25:C 20 May 2020 05:15:11.086 * DB saved on disk
25:C 20 May 2020 05:15:11.087 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:15:11.177 * Background saving terminated with success
1:S 20 May 2020 05:16:12.007 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:16:12.008 * Background saving started by pid 26
26:C 20 May 2020 05:16:12.013 * DB saved on disk
26:C 20 May 2020 05:16:12.014 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:16:12.109 * Background saving terminated with success
1:S 20 May 2020 05:18:10.856 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:18:10.857 * Background saving started by pid 27
27:C 20 May 2020 05:18:10.872 * DB saved on disk
27:C 20 May 2020 05:18:10.873 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:18:10.958 * Background saving terminated with success
1:S 20 May 2020 05:19:12.825 * 10 changes in 60 seconds. Saving...
1:S 20 May 2020 05:19:12.827 * Background saving started by pid 28
28:C 20 May 2020 05:19:12.833 * DB saved on disk
28:C 20 May 2020 05:19:12.834 * RDB: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:19:12.927 * Background saving terminated with success
1:S 20 May 2020 05:20:25.932 # User requested shutdown...
1:S 20 May 2020 05:20:25.932 * Calling fsync() on the AOF file.
1:S 20 May 2020 05:20:25.932 * Saving the final RDB snapshot before exiting.
1:S 20 May 2020 05:20:25.937 * DB saved on disk
1:S 20 May 2020 05:20:25.937 # Redis is now ready to exit, bye bye...
1:C 20 May 2020 05:20:34.372 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 20 May 2020 05:20:34.372 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 20 May 2020 05:20:34.372 # Configuration loaded
1:M 20 May 2020 05:20:34.399 * Node configuration loaded, I'm a90232a10cd0b4a49d10e2f829bf46e38687e266
1:M 20 May 2020 05:20:34.401 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 20 May 2020 05:20:34.401 # Server initialized
1:M 20 May 2020 05:20:34.401 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 20 May 2020 05:20:34.416 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 20 May 2020 05:20:34.416 * DB loaded from append only file: 0.000 seconds
1:M 20 May 2020 05:20:34.416 * Ready to accept connections
1:S 20 May 2020 05:20:34.426 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
1:S 20 May 2020 05:20:34.426 # Cluster state changed: ok
1:S 20 May 2020 05:20:35.429 * Connecting to MASTER 10.12.70.36:6379
1:S 20 May 2020 05:20:35.462 * MASTER <-> REPLICA sync started
1:S 20 May 2020 05:20:35.463 * Non blocking connect for SYNC fired the event.
1:S 20 May 2020 05:20:35.464 * Master replied to PING, replication can continue...
1:S 20 May 2020 05:20:35.466 * Trying a partial resynchronization (request c55c5d6e457e6f6c855d06f90d5e7c230731a3d0:1).
1:S 20 May 2020 05:20:35.468 * Full resync from master: b480823f3ac5dc08397e06f6a41a4c63f855c4f4:5431
1:S 20 May 2020 05:20:35.468 * Discarding previously cached master state.
1:S 20 May 2020 05:20:35.544 * MASTER <-> REPLICA sync: receiving 230 bytes from master
1:S 20 May 2020 05:20:35.545 * MASTER <-> REPLICA sync: Flushing old data
1:S 20 May 2020 05:20:35.546 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 20 May 2020 05:20:35.546 * MASTER <-> REPLICA sync: Finished with success
1:S 20 May 2020 05:20:35.546 * Background append only file rewriting started by pid 18
1:S 20 May 2020 05:20:35.573 * AOF rewrite child asks to stop sending diffs.
18:C 20 May 2020 05:20:35.573 * Parent agreed to stop sending diffs. Finalizing AOF...
18:C 20 May 2020 05:20:35.573 * Concatenating 0.00 MB of AOF diff received from parent.
18:C 20 May 2020 05:20:35.573 * SYNC append only file rewrite performed
18:C 20 May 2020 05:20:35.574 * AOF rewrite: 0 MB of memory used by copy-on-write
1:S 20 May 2020 05:20:35.663 * Background AOF rewrite terminated with success
1:S 20 May 2020 05:20:35.663 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
1:S 20 May 2020 05:20:35.663 * Background AOF rewrite finished successfully
1:S 20 May 2020 05:20:43.885 # User requested shutdown...
1:S 20 May 2020 05:20:43.886 * Calling fsync() on the AOF file.
1:S 20 May 2020 05:20:43.886 * Saving the final RDB snapshot before exiting.
1:S 20 May 2020 05:20:43.892 * DB saved on disk
1:S 20 May 2020 05:20:43.892 # Redis is now ready to exit, bye bye...
1:C 20 May 2020 05:20:56.211 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 20 May 2020 05:20:56.211 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 20 May 2020 05:20:56.211 # Configuration loaded
1:M 20 May 2020 05:20:56.242 * Node configuration loaded, I'm a90232a10cd0b4a49d10e2f829bf46e38687e266
1:M 20 May 2020 05:20:56.244 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 20 May 2020 05:20:56.244 # Server initialized
1:M 20 May 2020 05:20:56.244 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 20 May 2020 05:20:56.244 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 20 May 2020 05:20:56.244 * DB loaded from append only file: 0.000 seconds
1:M 20 May 2020 05:20:56.245 * Ready to accept connections