使用Apache Sqoop-1.4.7将所有表从MySQL导入Hive的问题

Question

[当我执行以下导入命令时，我试图使用Apache Sqoop CLI将所有表从MySQL数据库导入Hive数据库：

[hadoop@localhost bin]$ sqoop import-all-tables --connect jdbc:mysql://localhost/mysql --username root --password root

以某种方式导入失败，并且在输出末尾出现以下错误消息

20/05/09 23:06:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
.
.
.
20/05/09 23:06:38 ERROR tool.ImportAllTablesTool: Encountered IOException running import job:
java.io.IOException: Generating splits for a textual index column allowed only in case of
"-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter

在运行Sqoop-1.4.7之前，我已经安装并启动并运行了以下软件

[hduser@localhost ~]$ java -version
openjdk version "1.8.0_252" 
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

[hadoop@localhost ~]$ hadoop version
Hadoop 2.7.2
Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r
b165c4fe8a74265c792ce23f546c64604acf0e41
Compiled by jenkins on 2016-01-26T00:08Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /opt/hadoop/share/hadoop/common/hadoop-common-2.7.2.jar 

MySQL Server version: 5.6.48
Hive-1.2.2

基于此配置，如何将所有表成功导入到配置单元？

Answer 1

可以使用字符属性作为分割属性。根据您的错误消息

Generating splits for a textual index column allowed only in case of
"-Dorg.apache.sqoop.splitter.allow_text_splitter=true" property passed as a parameter

您只需要添加-Dorg.apache.sqoop.splitter.allow_text_splitter = true在这样的“ sqoop job”语句之后：

sqoop import-all-tables -Dorg.apache.sqoop.splitter.allow_text_splitter=true \
--connect jdbc:mysql://localhost/mysql \ 
--username root \
--password root