我在当前的项目中使用 Kafka 和 Avro,我决定升级我多年来的做事方式。
我总是从 AVDL 开始,因为我认为它是描述模式的最简洁的方式,然后使用 avro-maven-plugin 从中生成 Java 类。 当涉及 AVSC 架构文件时,我要么从 Java 类中手动检索它们,要么使用 exec-maven-plugin 在 Avro 工具上运行命令 idl2schemata 并生成它们。
但是,我注意到 avro-maven-plugin 发布了一个新的诱导目标:https://issues.apache.org/jira/browse/AVRO-2365 这个允许直接从 java 类生成 AVSC / AVPR:
mvn avro:help -Ddetail=true -Dgoal=induce
Maven plugin for Avro IDL and Specific API Compilers
avro:induce
Generate Avro files (.avsc and .avpr) from Java classes or interfaces
Available parameters:
allowNull (Default: false)
Whether to use ReflectData.AllowNull.
avroOutputDirectory (Default:
${project.build.directory}/generated-resources/avro)
Directory where to output Avro schemas (.avsc) or protocols (.avpr).
User property: avroOutputDirectory
encoding (Default: ${project.build.sourceEncoding})
The output encoding.
javaSourceDirectories (Default: ${basedir}/src/main/java)
The Java source directories.
User property: javaSourceDirectories
reflectDataImplementation
Override the default ReflectData implementation with an extension. Must be
a subclass of ReflectData.
User property: reflectDataImplementation
我的目标变得简单,仅使用 avro-maven-plugin 来:
我的 pom 会更干净,并且会保留在 idl 协议期间完成的模式自定义。
所以我带来了这样的东西:
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.11.1</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>idl-protocol</goal>
<goal>induce</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- idl-protocol goal - The source directory of avro files -->
<sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>
<!-- idl-protocol goal - The output directory of Java classes -->
<outputDirectory>${project.build.directory}/generated-sources/avro</outputDirectory>
<!-- idl-protocol goal - Customization -->
<createSetters>false</createSetters>
<stringType>String</stringType>
<!-- induce goal - The java source directory -->
<javaSourceDirectories>${project.build.directory}/generated-sources/avro</javaSourceDirectories>
<!-- induce goal - Directory where to output Avro schemas -->
<avroOutputDirectory>${project.build.directory}/generated-resources/avro</avroOutputDirectory>
</configuration>
</plugin>
但是,在演示模式上使用命令
mvn clean avro:idl-protocol avro:induce
:
@namespace("com.stack.overflow.kafka.event")
protocol CompensationProtocol {
record Question {
int memberId;
string title;
}
}
总是导致失败,执行诱导目标时
java.lang.ClassNotFoundException
:
[ERROR] Failed to execute goal org.apache.avro:avro-maven-plugin:1.11.1:induce (default-cli) on project compensation: Failed to load class rs.john.doe.git.demo.target.generated-sources.avro.com.stack.overflow.kafka.event.CompensationProtocol -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.avro:avro-maven-plugin:1.11.1:induce (default-cli) on project compensation: Failed to load class rs.john.doe.git.demo.target.generated-sources.avro.com.stack.overflow.kafka.event.CompensationProtocol
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:347)
...
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to load class rs.john.doe.git.demo.target.generated-sources.avro.com.stack.overflow.kafka.event.CompensationProtocol
at org.apache.avro.mojo.InduceMojo.loadClass (InduceMojo.java:184)
...
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:348)
Caused by: java.lang.ClassNotFoundException: rs.john.doe.git.demo.target.generated-sources.avro.com.stack.overflow.kafka.event.CompensationProtocol
at java.net.URLClassLoader.findClass (URLClassLoader.java:445)
at java.lang.ClassLoader.loadClass (ClassLoader.java:587)
at java.lang.ClassLoader.loadClass (ClassLoader.java:520)
at org.apache.avro.mojo.InduceMojo.loadClass (InduceMojo.java:182)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:124)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:115)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:115)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:115)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:115)
at org.apache.avro.mojo.InduceMojo.induceClasses (InduceMojo.java:115)
at org.apache.avro.mojo.InduceMojo.execute (InduceMojo.java:103)
...
Java 类照常正确生成,但 AVSC 文件未生成。
我尝试仅运行归纳目标,将类放入源文件夹而不是目标文件夹中,我什至使用公共 Avro 生成的 avro 类..例外总是存在。
为了找出异常的根本原因,我什至进入了插件源的
InduceMojo.class
,提取了相关源,使用maven插件依赖项构建了一个自定义项目..但我无法重现。
当提供正确的类名(此处
rs.john.doe.git.demo.target.generated-sources.avro.com.stack.overflow.kafka.event.CompensationProtocol
)时,我自己的类加载器能够加载它而不是抛出异常:
https://github.com/apache/avro/blob/master/lang/java/maven-plugin/src/main/java/org/apache/avro/mojo/InduceMojo.java#L182
在那个阶段,我或多或少放弃了使用诱导目标,但我仍然希望你们能有一个好主意来解决这个问题!
avro:idl-protocol,可以生成源码;
avro:induce
Generate Avro files (.avsc and .avpr) from Java classes or interfaces
所以,源代码必须在生成 *.avsc 之前编译。
pom.xml
<properties>
<java.version>17</java.version>
<avro.version>1.11.3</avro.version>
<avro.induce.skip>generate-resources</avro.induce.skip>
</properties>
<!-- ... -->
<artifactId>avro-maven-plugin</artifactId>
<version>${avro.version}</version>
<executions>
<execution>
<id>idl-protocol</id>
<phase>generate-sources</phase>
<goals>
<goal>idl-protocol</goal>
</goals>
<configuration>
<createSetters>false</createSetters>
<stringType>String</stringType>
<sourceDirectory>${project.basedir}/src/main/idl</sourceDirectory>
<outputDirectory>${project.build.sourceDirectory}</outputDirectory>
</configuration>
</execution>
<execution>
<id>induce</id>
<phase>${avro.induce.skip}</phase>
<goals>
<goal>induce</goal>
</goals>
<configuration>
<javaSourceDirectories>${project.build.sourceDirectory}</javaSourceDirectories>
<avroOutputDirectory>${project.build.directory}/generated-resources/avro</avroOutputDirectory>
</configuration>
</execution>
</executions>
然后一一运行这些命令:
mvn generate-sources
mvn compile -Davro.induce.skip=none
mvn generate-resources