sqoop import未识别显示ClassNotFountException错误的mysql表

问题描述 投票:0回答:1

我有一个mysql表“customers”,并尝试使用sqoop import将数据从mysql导入到hdfs位置。以下是我在我的机器上安装的版本:sqoop版本:1.4.6 hive版本:2.3.0 Hadoop版本:2.8.1

Sqoop import命令:

sqoop import --connect jdbc:mysql://localhost/localdb --username root --password mnbv@1234 --table customers -m 1 --target-dir /user/hduser/sqoop_import/customers1/

并显示在ClassNotFountException下面:

Tue Oct 31 09:57:21 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.17/10/31 09:57:21 INFO db.DBInputFormat: Using read commited transaction isolation             17/10/31 09:57:21 INFO mapred.MapTask: Processing split: 1=1 AND 1=1 
17/10/31 09:57:21 INFO mapred.LocalJobRunner: map task executor complete. 
17/10/31 09:57:22 WARN mapred.LocalJobRunner: job_local1437452057_0001java.lang.Exception: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class customers not found 
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class customers not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2216) 
at org.apache.sqoop.mapreduce.db.DBConfiguration.getInputClass(DBConfiguration.java:403)
at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.createDBRecordReader(DataDrivenDBInputFormat.java:237)
at org.apache.sqoop.mapreduce.db.DBInputFormat.createRecordReader(DBInputFormat.java:263)     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:515)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:758)     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270)     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 
Caused by: java.lang.ClassNotFoundException: Class customers not found     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2122)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2214)
... 12 more
17/10/31 09:57:22 INFO mapreduce.Job: Job job_local1437452057_0001 running in uber mode : false  17/10/31 09:57:22 INFO mapreduce.Job:  map 0% reduce 0%
17/10/31 09:57:22 INFO mapreduce.Job: Job job_local1437452057_0001 failed with state FAILED due to: NA  17/10/31 09:57:22 INFO mapreduce.Job: Counters: 0
17/10/31 09:57:22 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead  17/10/31 09:57:22 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 4.105 seconds (0 bytes/sec)
17/10/31 09:57:22 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead  17/10/31 09:57:22 INFO mapreduce.ImportJobBase: Retrieved 0 records.
17/10/31 09:57:22 ERROR tool.ImportTool: Error during import: Import job failed!

但是当我尝试使用sqoop命令列出表时,它工作正常并显示customers表。 Sqoop命令:

sqoop list-tables --connect jdbc:mysql://localhost/localdb --username root --password mnbv@1234;

输出显示正确,如下所示:

17/10/31 10:07:09 INFO manager.MySQLManager: Preparing to use a MySQL  streaming resultset.          
Tue Oct 31 10:07:09 IST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+,  5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not  using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true  and provide truststore for server certificate verification.                                                                                                customers

可能是什么问题,为什么sqoop从表导入不能识别mysql中的表。请帮助我。

提前致谢。

hadoop hive hdfs sqoop
1个回答
0
投票

使用--bindir选项并指向当前的工作目录。在--bindir选项下,给出$ SQOOP_HOME / lib的路径

很抱歉迟到的回复。根据最新评论更新答案。请检查以下内容:

  • 当sqoop进程在cmd:Writing jar file:/tmp/sqoop.../*.jar中运行时,需要检查jar文件
  • 使用import命令:sqoop import -fs local -jt local -libjars /tmp/..../*.jar --connect "jdbc:mysql://example.com/sqoop" --username <>--password <>--table customers
  • 验证生成的Java文件以进行sqoop导入。应该有 与正常导入的表格相同。 sqoop codegen --connect <""> --username <> --password <> --table customers

希望这可以帮助!

© www.soinside.com 2019 - 2024. All rights reserved.