Kafka 问题：MessageConversionException：无法将 GenericMessage 从 [java.lang.String] 转换为 [my_custom_model] [...]

Question

我有 Kafka 生产者和消费者服务器，当我尝试发送消息时，出现以下异常：

org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Endpoint handler details:
Method [public void com.mail.sender.service.senders.GmailConfirmationSenderService.confirmationMessageListener(com.mail.sender.dto.request.AccountRequest)]
Bean [com.mail.sender.service.senders.GmailConfirmationSenderService@24787445]; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot handle message; nested exception is org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [com.mail.sender.dto.request.AccountRequest] for GenericMessage [payload={"email":"[email protected]","username":"asdasd-mjeesh","confirmationTokenDetails":{"token":"3fd3c1ee-20ec-420b-8ee9-11d22cd7598e","createdAt":[2016,1,25,21,34,55],"expiredAt":[2023,1,8,18,19,6,661473300]}}, headers={kafka_offset=12, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@4008ea0f, kafka_timestampType=CREATE_TIME, kafka_receivedPartitionId=0, kafka_receivedTopic=mail_confirmation_message, kafka_receivedTimestamp=1673193848466, __TypeId__=[B@68a9f1ab, kafka_groupId=account_confirmation_group_id}]

我的生产者服务器的配置

@Configuration
public class KafkaProducerConfig {

    @Value("${spring.kafka.bootstrap-servers}")
    private String bootstrapServersUrl;

    public Map<String, Object> producerConfig() {
        Map<String, Object> props = new HashMap<>();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServersUrl);
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);
        props.put(JsonSerializer.TYPE_MAPPINGS, "accountRequest:com.confirmation_token.model.dto.request.outgoing.AccountRequest");
        return props;
    }

    @Bean
    public ProducerFactory<String, AccountRequest> producerFactory() {
        return new DefaultKafkaProducerFactory<>(producerConfig());
    }

    @Bean
    public KafkaTemplate<String, AccountRequest> kafkaTemplate(ProducerFactory<String, AccountRequest> producerFactory) {
        return new KafkaTemplate<>(producerFactory);
    }
}

我如何发送消息

@Service
@Slf4j
@RequiredArgsConstructor
public class EmailConfirmationSenderServiceCommunication implements ConfirmationSender {
    private final KafkaTemplate<String, AccountRequest> kafkaTemplate;

    @Override
    public void sendConfirmationToken(AccountRequest accountRequest) {
        kafkaTemplate.send("mail_confirmation_message", accountRequest);
        log.info("Confirmation token={} has been send", accountRequest);
    }
}

生产者端模型

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class AccountRequest {
    private String email;
    private String username;
    private ConfirmationTokenDetailsRequest confirmationTokenDetails;
}

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class ConfirmationTokenDetailsRequest {
    private String token;
    private LocalDateTime createdAt;
    private LocalDateTime expiredAt;
}

我的消费者服务器的配置

@Configuration
public class KafkaConsumerConfig {

    @Value("${spring.kafka.bootstrap-servers}")
    private String bootstrapServerUrl;

    public Map<String, Object> consumerConfig() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServerUrl);
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        props.put(JsonDeserializer.TYPE_MAPPINGS, "accountRequest:com.mail.sender.dto.request.AccountRequest");
        return props;
    }

    @Bean
    public ConsumerFactory<String, AccountRequest> consumerFactory() {
        return new DefaultKafkaConsumerFactory<>(consumerConfig());
    }

    @Bean
    public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, AccountRequest>> listenerContainerFactory(
            ConsumerFactory<String, AccountRequest> consumerFactory) {
        ConcurrentKafkaListenerContainerFactory<String, AccountRequest> listenerContainerFactory =
                new ConcurrentKafkaListenerContainerFactory<>();
        listenerContainerFactory.setConsumerFactory(consumerFactory);
        return listenerContainerFactory;
    }
}

主题配置

@Configuration
public class KafkaTopicConfig {

    @Value("${kafka.topic.names.account.confirmation}")
    private String accountMailConfirmationTopicName;

    @Bean
    public NewTopic accountConfirmationTopic() {
        return TopicBuilder
                .name(accountMailConfirmationTopicName)
                .build();
    }
}

我如何尝试聆听消息

@Service
@Slf4j
@RequiredArgsConstructor
public class GmailConfirmationSenderService implements EmailSender<AccountRequest> {
    private final String accountConfirmationTemplate;
    private final ApplicationModelValidator applicationModelValidator;
    private final JavaMailSender mailSender;

    @Value("${confirmation.link.template}")
    private String confirmationLink;

    @KafkaListener(topics = "mail_confirmation_message", groupId = "account_confirmation_group_id")
    public void confirmationMessageListener(AccountRequest accountRequest) {
        String validationViolations = applicationModelValidator.validate(accountRequest);
        if (!validationViolations.isBlank()) {
            log.error(validationViolations);
            throw new ModelValidationException(validationViolations);
        }

        String userConfirmationLink = String.format(
                accountConfirmationTemplate,
                accountRequest.getUsername(),
                confirmationLink + accountRequest.getConfirmationTokenDetails().getToken());
        this.send(accountRequest, userConfirmationLink);
    }
    
    ...
}

消费侧模型

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class AccountRequest {
    @Email
    private String email;

    @Length(min = 8, max = 40)
    private String username;

    @NotNull
    @Valid
    private ConfirmationTokenDetailsRequest confirmationTokenDetails;
}

@Builder
@Data
@AllArgsConstructor
@NoArgsConstructor
public class ConfirmationTokenDetailsRequest {
    @NotBlank
    private String token;

    @PastOrPresent
    private LocalDateTime createdAt;

    @Future
    private LocalDateTime expiredAt;
}

*消费者的模型中有用于验证的注释，但它们不影响流程

Answer 1

您的侦听器正在使用默认（字符串）解串器（可能来自引导的自动配置工厂）。

listenerContainerFactory

- 由于您使用的是非标准工厂名称，因此需要在

@KafkaListener

上指定它（或将工厂 bean 名称更改为

kafkaListenerContainerFactory

）。

Answer 2

或者只使用 JsonSerializer/JsonDeserializer

spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer