Kafka Stream Rebalancing:从REBALANCING到ERROR的状态转变

问题描述 投票:1回答:1

我有4个主题,包括单个分区和3个应用程序实例。我尝试通过编写自定义PartitionGrouper来实现可伸缩性,这将创建3个任务,如下所示:

1st instance-topic1,partition0,topic4,partition0

第二个实例 - topic2,partition0

第3个实例 - topic3,partition0

我将NUM_STANDBY_REPLICAS_CONFIG配置为1,因为它将在本地维护状态(也是为了消除invalidstatestore exception)。

以上设置适用于两个实例。当我把它增加到三个实例时,我开始面临w.r.t重新平衡的问题。

StickyTaskAssignor:58 - Unable to assign 1 of 1 standby tasks for task [1009710637_0]. There is not enough available capacity. You should increase the number of threads and/or application instances to maintain the requested number of standby replicas.
    [INFO ] 2017-12-25 20:05:42.221 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:888 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] State transition from PARTITIONS_REVOKED to PARTITIONS_ASSIGNED.
    [INFO ] 2017-12-25 20:05:42.221 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] KafkaStreams:268 - stream-client [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449] State transition from REBALANCING to REBALANCING.
    [INFO ] 2017-12-25 20:05:42.276 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:195 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] partition assignment took 55 ms.
    current active tasks: [1009710637_0]
    current standby tasks: [1240464215_0, 1833680710_0]
    previous active tasks: []
    [INFO ] 2017-12-25 20:05:42.631 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:939 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] Shutting down
    [INFO ] 2017-12-25 20:05:42.631 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:888 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] State transition from PARTITIONS_ASSIGNED to PENDING_SHUTDOWN.
    [INFO ] 2017-12-25 20:05:42.633 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] KafkaProducer:972 - Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
    [INFO ] 2017-12-25 20:05:42.638 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:972 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] Stream thread shutdown complete
    [INFO ] 2017-12-25 20:05:42.638 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] StreamThread:888 - stream-thread [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD.
    [WARN ] 2017-12-25 20:05:42.638 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] KafkaStreams:343 - stream-client [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449] All stream threads have died. The Kafka Streams instance will be in an error state and should be closed.
    [INFO ] 2017-12-25 20:05:42.638 [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449-StreamThread-1] KafkaStreams:268 - stream-client [app-03-cfaf7841-dc19-4ee4-9d05-ae4928c21449] State transition from REBALANCING to ERROR.
apache-kafka-streams
1个回答
0
投票

我假设你的PartitionGrouper打破了一些东西。编写正确的自定义分区分组器非常困难,因为您需要了解很多关于Kafka Streams的内部信息。因此,首先不建议使用它。

错误本身意味着,由于没有足够的线程,因此无法成功将StandbyTask分配给线程。一般来说,想法是StandbyTask不能分配给运行相应“活动”任务的线程或同一StandbyTasks的另一个副本:它不会增加容错但只会浪费内存,就好像线程死了一样,全部任务死了。

为什么你特别得到这个错误是不清楚的(快乐调试:))。

但是,对于您的用例,您应该只是启动订阅单个主题的不同应用程序实例,并使用不同的application.id来扩展您的应用程序。

© www.soinside.com 2019 - 2024. All rights reserved.