添加新分片时租赁表不会更新，导致 KCL 中的工作人员过时

Question

我正在使用 amazon-kclpy 版本“2.1.3”（使用 MultiLangDaemon 的 python 库）。

根据 aws 文档（here），当我为我的流添加新分片或当新工作线程（在我的例子中是 Kubernetes 中的 pod）启动时，应该发生重新分片事件。

重新分片后，我应该在租赁表中看到新记录。

但是，在这两种情况下，我在租赁表中没有看到任何新记录，导致添加的新工作人员过时并进入睡眠周期......

2023-12-24 17:46:34,750 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - Initializing LeaseCoordinator attempt 1 
2023-12-24 17:46:37,860 [multi-lang-daemon-0000] INFO  s.a.k.leases.LeaseCleanupManager [NONE] - Starting lease cleanup thread. 
2023-12-24 17:46:37,861 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - Starting LeaseCoordinator 
2023-12-24 17:46:37,861 [pool-15-thread-1] INFO  s.a.k.leases.LeaseCleanupManager [NONE] - Number of pending leases to clean before the scan : 0 
2023-12-24 17:46:37,926 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - Scheduling periodicShardSync 
2023-12-24 17:46:37,928 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - Initialization complete. Starting worker loop. 
2023-12-24 17:46:38,026 [multi-lang-daemon-0000] INFO  s.a.k.c.DeterministicShuffleShardSyncLeaderDecider [NONE] - Elected leaders: fe86945c-96d7-423e-a157-97c5c6ad6f79 
2023-12-24 17:47:05,036 [multi-lang-daemon-0000] INFO  s.a.k.c.DiagnosticEventLogger [NONE] - Current thread pool executor state: ExecutorStateEvent(executorName=SchedulerThreadPoolExecutor, currentQueueSize=0, activeThreads=0, coreThreads=0, leasesOwned=0, largestPoolSize=0, maximumPoolSize=2147483647) 
2023-12-24 17:47:35,044 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - No activities assigned 
2023-12-24 17:47:35,045 [multi-lang-daemon-0000] INFO  s.a.k.c.DiagnosticEventLogger [NONE] - Current thread pool executor state: ExecutorStateEvent(executorName=SchedulerThreadPoolExecutor, currentQueueSize=0, activeThreads=0, coreThreads=0, leasesOwned=0, largestPoolSize=0, maximumPoolSize=2147483647) 
2023-12-24 17:47:35,045 [multi-lang-daemon-0000] INFO  s.a.kinesis.coordinator.Scheduler [NONE] - Sleeping ... 

我不确定我缺少什么才能导致新的 pod 被分配给这些新分片，以便 pod 真正处理记录。

这是权限问题吗？也许有 python 代码？

Answer 1

我无法说出问题的根源，但当我用 pod 名称（作为 KCL 运行命令的一部分读取的属性文件）替换属性文件中的worker_id 属性后，我发现问题就停止了。