Cassandra节点频繁关闭。 InetAddress /XX.XX.XX.XX现在为DOWN

Question

从cassandra读取数据时，查询超时，下面是调试日志

DEBUG [GossipTasks:1] 2019-11-29 00:21:05,940 Gossiper.java:353 - Convicting /XX.XX.XX.XX with status NORMAL - alive true
        INFO  [GossipTasks:1] 2019-11-29 00:21:05,941 Gossiper.java:1034 - InetAddress /XX.XX.XX.XX is now DOWN
        DEBUG [GossipTasks:1] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:205 - Enqueuing socket close for /XX.XX.XX.XX
        DEBUG [GossipTasks:1] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:205 - Enqueuing socket close for /XX.XX.XX.XX
        DEBUG [GossipTasks:1] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:205 - Enqueuing socket close for /XX.XX.XX.XX
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Large] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:411 - Socket to /XX.XX.XX.XX closed
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Small] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:411 - Socket to /XX.XX.XX.XX closed
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Gossip] 2019-11-29 00:21:05,941 OutboundTcpConnection.java:411 - Socket to /XX.XX.XX.XX closed
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Gossip] 2019-11-29 00:21:08,942 OutboundTcpConnection.java:425 - Attempting to connect to /XX.XX.XX.XX
        INFO  [HANDSHAKE-/XX.XX.XX.XX] 2019-11-29 00:21:08,943 OutboundTcpConnection.java:561 - Handshaking version with /XX.XX.XX.XX
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Gossip] 2019-11-29 00:21:13,943 OutboundTcpConnection.java:411 - Socket to /XX.XX.XX.XX closed
        INFO  [HANDSHAKE-/XX.XX.XX.XX] 2019-11-29 00:21:13,943 OutboundTcpConnection.java:570 - Cannot handshake version with /XX.XX.XX.XX
        INFO  [HANDSHAKE-/XX.XX.XX.XX] 2019-11-29 00:21:13,944 OutboundTcpConnection.java:561 - Handshaking version with /XX.XX.XX.XX
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Gossip] 2019-11-29 00:21:18,688 OutboundTcpConnection.java:533 - Done connecting to /XX.XX.XX.XX
        DEBUG [MessagingService-Outgoing-/XX.XX.XX.XX-Small] 2019-11-29 00:21:18,705 OutboundTcpConnection.java:425 - Attempting to connect to /XX.XX.XX.XX
        INFO  [HANDSHAKE-/XX.XX.XX.XX] 2019-11-29 00:21:18,706 OutboundTcpConnection.java:561 - 

Answer 1

经过长期调查，红外小组在存储级别发现了问题。一个坏磁盘上出现了拥塞。绕过磁盘修复了问题。