在 Kubernetes 上运行的 Apache Ignite 客户端无法加入部署在 Kubernetes 外部的 Ignite 服务器的集群

Question

我们在一组虚拟机上部署了旧版 Apache Ignite 服务器 (v2.9.1)，并尝试从部署在 Kubernetes 上的服务通过胖客户端连接到服务器。

我们尝试过这个配置

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd">
    <bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
        <!-- Set to true to enable distributed class loading for examples, default is false. -->
        <property name="peerClassLoadingEnabled" value="false"/>
        <!-- Enable client mdoe. -->
        <property name="clientMode" value="true"/>
        <!-- Work Dir -->
        <property name="workDirectory" value="/tmp"/>
        <!-- Network Segmentation (Split Brain) -->
        <property name="segmentationPolicy" value="NOOP"/>
        <property name="segmentCheckFrequency" value="60000"/>

        <property name="clientFailureDetectionTimeout" value="60000"/>

        <!-- Enable task execution events for examples. -->
        <property name="includeEventTypes">
            <list>
            </list>
        </property>

        <property name="discoverySpi">
            <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="networkTimeout" value="15000"/>
                <property name="socketTimeout" value="15000"/>
                <property name="ackTimeout" value="15000"/>
                <property name="statisticsPrintFrequency" value="30"/>
                <property name="ipFinder">
                    <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                        <property name="addresses" value="server-1.my-domain.com,server-2.my-domain.com,server-3.my-domain.com"/>
                    </bean>
                </property>
            </bean>
        </property>
        <property name="communicationSpi">
            <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
                <property name="forceClientToServerConnections" value="true"/>
            </bean>
        </property>
    </bean>
    <bean id="conversionService"
            class="org.springframework.context.support.ConversionServiceFactoryBean"/>
</beans>

但因以下错误而失败， 客户端日志：

SEVERE: Failed to send message: null
java.io.IOException: Failed to get acknowledge for message: TcpDiscoveryClientMetricsUpdateMessage [super=TcpDiscoveryAbstractMessage [sndNodeId=null, id=57c6bd40b81-6c95ce16-6b3a-4046-80fa-6cdaac6c4831, verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, isClient=true]]
        at org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1471)
        at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
...
...
SEVERE: Got exception while starting (will rollback startup routine).
class org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1921)
        at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1279)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2052)
        at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1698)
        at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1114)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:634)
        at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:575)
        at org.apache.ignite.Ignition.getOrStart(Ignition.java:419)
        at my.IgniteClientService.getIgnite(PRIgniteService.java:575)
        at my.IgniteClientService.startIgniteService(IgniteClientService.java:926)
        at my.IgniteClientService.<init>(IgniteClientService.java:100)
        at my.IgniteClientService.<init>(IgniteClientService.java:78)
        at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=15000, ackTimeout=15000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@68c517], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=0, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
        at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:302)
        at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:974)
        at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1916)
        ... 32 more
Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to connect to cluster, connection failed and failed to reconnect.
        at org.apache.ignite.spi.discovery.tcp.ClientImpl$Reconnector.body(ClientImpl.java:1691)
        at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)

服务器日志：

[12:00:51,574][WARNING][disco-event-worker-#323][GridDiscoveryManager] Node FAILED: TcpDiscoveryNode [id=6c95ce16-6b3a-4046-80fa-6cdaac6c4831, consistentId=6c95ce16-6b3a-4046-80fa-6cdaac6c4831, addrs=ArrayList [10.244.8.141, 127.0.0.1], sockAddrs=HashSet [/10.244.8.141:0, /127.0.0.1:0], discPort=0, order=2126, intOrder=1071, lastExchangeTime=1696593572069, loc=false, ver=2.9.1#20201203-sha1:adcce517, isClient=true]

这里10.244.8.141是POD的IP地址。 有类似设置工作的人请帮助我们！！

Answer 1

服务器节点日志中是否还有其他错误？通过整个集群上的日志来检查它们。