为什么kafka-avro-console-producer不支持该字段的默认值？

Question

虽然默认是为字段定义的，但kafka-avro-console-producer完全忽略它：

$ kafka-avro-console-producer --broker-list localhost:9092 --topic test-avro \
--property schema.registry.url=http://localhost:8081 --property \
value.schema='{"type":"record","name":"myrecord1","fields": \
[{"name":"f1","type":"string"},{"name": "f2", "type": "int", "default": 0}]}'

{"f1": "value1"}

org.apache.kafka.common.errors.SerializationException: Error 
deserializing json {"f1": "value1"} to Avro of schema 
{"type":"record","name":"myrecord1","fields": 
[{"name":"f1","type":"string"},{"name":"f2","type":"int","default":0}]}
Caused by: org.apache.avro.AvroTypeException: Expected int. Got END_OBJECT
    at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:698)
    at org.apache.avro.io.JsonDecoder.readInt(JsonDecoder.java:172)
    at org.apache.avro.io.ValidatingDecoder.readInt(ValidatingDecoder.java:83)
    at org.apache.avro.generic.GenericDatumReader.readInt(GenericDatumReader.java:511)
    at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:182)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
    at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:240)
    at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:230)
    at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:174)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
    at io.confluent.kafka.formatter.AvroMessageReader.jsonToAvro(AvroMessageReader.java:213)
    at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:180)
    at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:54)
    at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)

那么如何使用才能接受默认值？顶级配置设置为"BACKWARD"兼容级别检查，虽然我不相信这与问题有任何关系。这个模式是版本2，版本1只用f1字段定义，但正如我所说，我认为这不重要。

Answer 1

正如Avro spec中所定义的那样

default：此字段的默认值，用于读取缺少此字段的实例

因此，生产者仍然需要提供该领域。

我不确定在使用Avro控制台生产者时是否可以完全排除某个字段，因为即使您将该字段视为Giorgos显示的字段，您仍需要明确设置它。

Answer 2

该错误表示该消息与您定义的Avro架构不兼容。据我所知，你想允许字段null的f2值。为此，您需要将value.schema更改为（请注意"type"的定义）：

value.schema='{"type":"record","name":"myrecord1","fields": [{"name":"f1","type":"string"},{"name": "f2", "type": ["null", "int"], "default": 0}]}' 

但你仍然需要用空值定义f2键。以下应该为您解决问题：

kafka-avro-console-producer --broker-list localhost:9092 --topic test-avro \ 
    --property schema.registry.url=http://localhost:8081 \ 
    --property value.schema='{"type":"record","name":"myrecord1","fields": [{"name":"f1","type":"string"},{"name": "f2", "type": ["null", "int"], "default": 0}]}'

{"f1":"value1","f2":null} 

你可以使用kafka-avro-console-consumer确认这是有效的：

kafka-avro-console-consumer --bootstrap-server localhost:9092 --topic test-avro --from-beginning
{"f1":"value1","f2":null}
^CProcessed a total of 1 messages