KSQL表中如何有重复的ID?

问题描述 投票:1回答:1

我正在使用Confluent社区和Postgres数据库,并且遇到了以下问题。

事件很好地流入kafka,并创建了主题。我创建了一个基于主题的流,并重新设置了密钥,因为密钥为null。

在重新生成密钥的流所基于的新主题之外,我创建了一个表。目标是拥有一个不断更新的对象表(此处为类别)。

事实是,当我在数据库中进行手动UPDATE时,表永远不会使用新数据进行更新。这些行只是像流一样不断添加。当然,我再次进行了选择,因为我知道当我们仍在运行查询时会显示“更新”行。

ksql> select * from categories;
1568287458487 | 1 | 1 | Beverages | Soft drinks, coffees, teas, beers, and ales
1568287458487 | 2 | 2 | Condiments | Sweet and savory sauces, relishes, spreads, and seasonings
1568287458488 | 3 | 3 | Confections | Desserts, candies, and sweet breads
1568287458488 | 4 | 4 | Dairy Products | Cheeses
1568287458488 | 5 | 5 | Grains/Cereals | Breads, crackers, pasta, and cereal
1568287458488 | 6 | 6 | Meat/Poultry | Prepared meats
1568287458489 | 7 | 7 | Produce | Dried fruit and bean curd
1568287458489 | 8 | 8 | Seafood | Seaweed and fish
1568288647248 | 8 | 8 | Seafood2 | Seaweed and fish
1568290562250 | 1 | 1 | asdf | Soft drinks, coffees, teas, beers, and ales
1568296165250 | 8 | 8 | Seafood3 | Seaweed and fish
1568296704747 | 8 | 8 | Seafood4 | Seaweed and fish
^CQuery terminated
ksql> select * from categories;
1568287458487 | 1 | 1 | Beverages | Soft drinks, coffees, teas, beers, and ales
1568287458487 | 2 | 2 | Condiments | Sweet and savory sauces, relishes, spreads, and seasonings
1568287458488 | 3 | 3 | Confections | Desserts, candies, and sweet breads
1568287458488 | 4 | 4 | Dairy Products | Cheeses
1568287458488 | 5 | 5 | Grains/Cereals | Breads, crackers, pasta, and cereal
1568287458488 | 6 | 6 | Meat/Poultry | Prepared meats
1568287458489 | 7 | 7 | Produce | Dried fruit and bean curd
1568287458489 | 8 | 8 | Seafood | Seaweed and fish
1568288647248 | 8 | 8 | Seafood2 | Seaweed and fish
1568290562250 | 1 | 1 | asdf | Soft drinks, coffees, teas, beers, and ales
1568296165250 | 8 | 8 | Seafood3 | Seaweed and fish
1568296704747 | 8 | 8 | Seafood4 | Seaweed and fish
^CQuery terminated
ksql> 

postgres中的类别表:

CREATE TABLE categories (
    category_id smallint NOT NULL,
    category_name character varying(15) NOT NULL,
    description text
);

KSQL中的类别表:

ksql> describe extended categories;

Name                 : CATEGORIES
Type                 : TABLE
Key field            : CATEGORY_ID_ST
Key format           : STRING
Timestamp field      : Not set - using <ROWTIME>
Value format         : AVRO
Kafka topic          : categories_rk (partitions: 1, replication: 1)

 Field          | Type                      
--------------------------------------------
 ROWTIME        | BIGINT           (system) 
 ROWKEY         | VARCHAR(STRING)  (system) 
 CATEGORY_ID_ST | VARCHAR(STRING)           
 CATEGORY_NAME  | VARCHAR(STRING)           
 DESCRIPTION    | VARCHAR(STRING)           
 MESSAGETOPIC   | VARCHAR(STRING)           
 MESSAGESOURCE  | VARCHAR(STRING)           
--------------------------------------------

应该具有唯一ROWKEY的表如何继续使用相同的ROWKEY添加更多的'update'行?

我实际上期望表格显示类别的总是最新列表,如https://www.youtube.com/watch?v=DPGn-j7yD68&list=PLa7VYi0yPIH2eX8q3mPpZAn3qCS1eDX8W&index=9中所述:

“表是事件的实例化视图,每个键只有最新值”。但是也许我误会了?

我正在使用Confluent社区和Postgres数据库,并且遇到了以下问题。事件很好地流入kafka,并创建了主题。我从...

apache-kafka confluent ksqldb
1个回答
0
投票

随着新数据的到来,KSQL中的表会不断更新。写入表的行的输出主题称为更改日志:它是表更改的不可变日志。如果特定密钥被多次更新,则输出主题将包含同一密钥的多个消息。每个新值将替换最后一个。

© www.soinside.com 2019 - 2024. All rights reserved.