Flink中的java.lang.NoSuchMethodError

问题描述 投票:1回答:6

我试图使用以下方法读取文件:

final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet<String> line = env.readTextFile("file:///pathtofile/myfile.txt");

我收到以下错误:

java.lang.NoSuchMethodError: org.apache.flink.api.common.io.DelimitedInputFormat: method <init>(Lorg/apache/flink/core/fs/Path;)V not found

我使用的是flink 1.3.2版,java版“1.8.0_91”

java file apache-flink
6个回答
2
投票

与依赖项存在冲突。 Apache Flink默认将许多类加载到其类路径中。

请阅读本文https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/debugging_classloading.html的最后一节

使用maven-shade-plugin解决与Flink的依赖冲突

Apache Flink默认将许多类加载到其类路径中。如果用户经常使用Flink正在使用的库的不同版本

IllegalAccessExceptions

要么

NoSuchMethodError

是结果。

所以,我建议你使用你的pom.xml并使用maven-shade-plugin并添加正确的重定位,就像我们在示例中所做的那样

<relocation>
  <pattern>org.codehaus.plexus.util</pattern>
  <shadedPattern>org.shaded.plexus.util</shadedPattern>
  <excludes>
    <exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
    <exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
  </excludes>
</relocation>

2
投票

您是否在IntelliJ或Dashboard中收到此错误,如果您在IntelliJ中收到此错误,请确保在pom.xml中使用相同的Flink版本,并在构建中添加依赖关系着色,如下所示

 <build>

        <plugins>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>

                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                    <compilerId>jdt</compilerId>
                </configuration>

                <dependencies>
                    <dependency>
                        <groupId>org.eclipse.tycho</groupId>
                        <artifactId>tycho-compiler-jdt</artifactId>
                        <version>0.21.0</version>
                    </dependency>
                </dependencies>
            </plugin>


            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <relocations>
                                <relocation>
                                    <pattern>org.codehaus.plexus.util</pattern>
                                    <shadedPattern>org.shaded.plexus.util</shadedPattern>
                                    <excludes>
                                        <exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude>
                                        <exclude>org.codehaus.plexus.util.xml.pull.*</exclude>
                                    </excludes>
                                </relocation>
                            </relocations>
                        </configuration>
                    </execution>
                </executions>
            </plugin>


        </plugins>


</build>>

确保在进行更改后在终端中运行maven clean install。另一方面,如果你只在仪表板中有这个问题而不是在intelliJ中,那么看看here


0
投票

您需要检查构建路径,确保libs存在并正确导入它们


0
投票

错误“java.lang.NoSuchMethodError”的一个可能原因是当我们使用不同版本的flink时,我们在系统上安装了什么。对我来说,我有Flink 1.3.2,而我使用的版本是1.1.1。所以我更新了我的pom文件以获得相同的版本。


0
投票

对于那些正在努力解决上述问题的人,在使用qazxsw poi时,这里是我能够成功构建的整个pom:

Flink 1.3.2

请在<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-assembly-plugin</artifactId> <version>2.4.1</version> <configuration> <!-- get all project dependencies --> <descriptorRefs> <descriptorRef>jar-with-dependencies</descriptorRef> </descriptorRefs> <!-- MainClass in mainfest make a executable jar --> <archive> <manifest> <mainClass>FlinktoLambda</mainClass> </manifest> </archive> </configuration> <executions> <execution> <id>make-assembly</id> <!-- bind to the packaging phase --> <phase>package</phase> <goals> <goal>single</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-jar-plugin</artifactId> <configuration> <archive> <manifest> <mainClass>FlinktoLambda</mainClass> </manifest> </archive> </configuration> </plugin> <!--added newly--> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.1</version> <configuration> <source>1.8</source> <target>1.8</target> <compilerId>jdt</compilerId> </configuration> <dependencies> <dependency> <groupId>org.eclipse.tycho</groupId> <artifactId>tycho-compiler-jdt</artifactId> <version>0.21.0</version> </dependency> </dependencies> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>3.1.0</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>FlinktoLambda</mainClass> </transformer> <transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer"> <resource>reference.conf</resource> </transformer> </transformers> <relocations> <relocation> <pattern>org.codehaus.plexus.util</pattern> <shadedPattern>org.shaded.plexus.util</shadedPattern> <excludes> <exclude>org.codehaus.plexus.util.xml.Xpp3Dom</exclude> <exclude>org.codehaus.plexus.util.xml.pull.*</exclude> </excludes> </relocation> </relocations> </configuration> </execution> </executions> </plugin> </plugins> </build> <name>my-app</name> <url>http://maven.apache.org</url> <dependencyManagement> <dependencies> <dependency> <groupId>FlinktoLambda</groupId> <artifactId>my-app</artifactId> <exclusions> <exclusion> <groupId>com.typesafe.akka</groupId> <artifactId>akka-remote_2.10</artifactId> </exclusion> <exclusion> <groupId>com.typesafe.akka</groupId> <artifactId>akka-actor_2.10</artifactId> </exclusion> <exclusion> <groupId>com.typesafe.akka</groupId> <artifactId>akka-slf4j_2.10</artifactId> </exclusion> </exclusions> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-table_2.10</artifactId> <version>1.3.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.3.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.10</artifactId> <version>1.3.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.10</artifactId> <version>1.3.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-scala_2.10</artifactId> <version>1.3.2</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-scala_2.10</artifactId> <version>1.3.2</version> </dependency> </dependencies> 插件中相应更改您的主类。


0
投票

此错误的另一个原因可能是Flink与您的应用程序代码(或使用scala的应用程序依赖项)之间的scala版本不匹配。

例如,在我的情况下,我使用的是Flink 1.7.1,我必须将我的scala依赖项从2.11更新为2.12;我更新了相关依赖项的artifcatId如下:从shadeflink-scala_2.11flink-scala_2.12flink-table_2.11等。

有关更多信息,请参阅flink-table_2.12

© www.soinside.com 2019 - 2024. All rights reserved.