Kafka 连接插件

Question

正如您在下面的 docker-compose.yaml 文件中看到的，我已经配置了 kafka-connect debezium。您可以看到字段吗？ ：CONNECT_PLUGINS_DIR 和 CONNECT_PLUGIN_PATH 为空。因此，我们必须在连接器插件级别有一个空字段。也就是说：

curl -s localhost:8083/connector-plugins | jq .

应该给出一个空结果。但是当我们这样做时，我们得到这个：

docker-compose-demo git:(main) ✗ curl -s localhost:8083/connector-plugins | jq.[{"class": "org.apache.kafka.connect.mirror.MirrorCheckpointConnector","type": "source","version": "3.6.1"},{"class": "org.apache.kafka.connect.mirror.MirrorHeartbeatConnector","type": "source","version": "3.6.1"},{"class": "org.apache.kafka.connect.mirror.MirrorSourceConnector","type": "source","version": "3.6.1"}]

我对 connect-distributed.properties 文件进行了检查，我们看到插件路径是空的。

➜ docker-compose-demo git:(main) ✗ docker exec -ti kconnect cat /kafka/config/connect-distributed.properties | grep plugin.path

plugin.path=/home

此外，使用命令

docker-compose-demo git:(main) ✗ docker exec -ti kconnect ls /home

我们看到

/home

是空的。我们不应该有任何插件。 这些插件从哪里来以及如何删除它们？

这些插件从哪里来以及如何删除它们？

您将在下面看到 docker-compose.yaml。

version: '2'
services:
#1-Un cluster kafka avec debezium/kafka:2.4.1.Final, on a 3 brokers kafka
    kafka-1:
        image: debezium/kafka:2.4
        container_name: kafka-1
        ports:
            - 9092:9092
        environment:
            - CLUSTER_ID=FFFFFF
            - BROKER_ID=1
            - KAFKA_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
            - KAFKA_LISTENERS=INTERNAL://:29092,EXTERNAL://:39092,CONTROLLER://kafka-1:9092
            - KAFKA_ADVERTISED_LISTENERS=INTERNAL://kafka-1:29092,EXTERNAL://localhost:39092
            - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
            - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
    kafka-2:
        image: debezium/kafka:2.4
        container_name: kafka-2
        ports:
            - 9093:9093
        environment:
            - CLUSTER_ID=FFFFFF
            - BROKER_ID=2
            - KAFKA_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
            - KAFKA_LISTENERS=INTERNAL://:29093,EXTERNAL://:39093,CONTROLLER://kafka-2:9093
            - KAFKA_ADVERTISED_LISTENERS=INTERNAL://kafka-2:29093,EXTERNAL://localhost:39093
            - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
            - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
    kafka-3:
        image: debezium/kafka:2.4
        container_name: kafka-3
        ports:
            - 9094:9094
        environment:
            - CLUSTER_ID=FFFFFF
            - BROKER_ID=3
            - KAFKA_CONTROLLER_QUORUM_VOTERS=1@kafka-1:9092,2@kafka-2:9093,3@kafka-3:9094
            - KAFKA_LISTENERS=INTERNAL://:29094,EXTERNAL://:39094,CONTROLLER://kafka-3:9094
            - KAFKA_ADVERTISED_LISTENERS=INTERNAL://kafka-3:29094,EXTERNAL://localhost:39094
            - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT
            - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
    kconnect:
        image: debezium/connect:2.5
        container_name: kconnect
        ports:
            - 8083:8083
        environment:
            CONFIG_STORAGE_TOPIC: my_connect_configs
            OFFSET_STORAGE_TOPIC: my_connect_offsets
            STATUS_STORAGE_TOPIC: my_connect_statuses
            BOOTSTRAP_SERVERS: kafka-1:29092,kafka-2:29093,kafka-3:29094
            CONNECT_PLUGINS_DIR: 
            CONNECT_PLUGIN_PATH:

        depends_on:
            - kafka-1
            - kafka-2
            - kafka-3

Answer 1

应该给出空结果

这是不正确的。 Kafka 附带了 MirrorMaker2，它基于 Connect 框架。