凤凰表上的配置单元查询抛出ColumnNotFoundException

Question

我使用hbase和hive（hive-server2）运行EMR集群。

1. 我的ETL管道使用数据创建并填充Phoenix表。

   CREATE TABLE IF NOT EXISTS UNMAPPED_FACTS (
     ACCOUNT VARCHAR NOT NULL,
     CONTAINER VARCHAR NOT NULL,
     UID_TYPE VARCHAR NOT NULL,
     UID VARCHAR NOT NULL,
     TS_EPOCH_MILLIS BIGINT NOT NULL,
     DP_KEY VARCHAR NOT NULL,
     DP_VALUE VARCHAR NOT NULL
       CONSTRAINT pk PRIMARY KEY (ACCOUNT, CONTAINER, UID_TYPE, UID, TS_EPOCH_MILLIS, DP_KEY)
   ) SALT_BUCKETS = 256
   

2. 然后，我在蜂巢元存储中创建一个指向我的凤凰表的EXTERNAL表。

   CREATE EXTERNAL TABLE IF NOT EXISTS unmapped_facts (
       account STRING,
       container STRING,
       uid_type STRING,
       uid STRING,
       ts_epoch_millis BIGINT,
       dp_key STRING,
       dp_value STRING
   )
   STORED BY 'org.apache.phoenix.hive.PhoenixStorageHandler'
   TBLPROPERTIES (
   "phoenix.table.name" = "UNMAPPED_FACTS",
   "phoenix.zookeeper.quorum" = "${zookeeper_host}",
   "phoenix.zookeeper.znode.parent" = "/hbase",
   "phoenix.zookeeper.client.port" = "2181",
   "phoenix.rowkeys" = "ACCOUNT, CONTAINER, UID_TYPE, UID, TS_EPOCH_MILLIS, DP_KEY"
   );
   

3. 然后我在其上运行hive查询，例如：

   select * from unmapped_facts limit 10
   

[当我使用EMR 5.6.0（Phoenix 4.9.0-HBase-1.2和HBase 1.2.3）时，所有这些都起作用。

现在我将EMR升级到最新的5.7.0（Phoenix 4.11.0-HBase-1.3和HBase 1.3.1），现在第1步和第2步可以正常工作，但是执行查询会引发异常（请参见下文）。我可以使用sqlline在我的凤凰表上执行sql查询，而不会出现任何问题。

非常感谢您提供调试问题的帮助。

Caused by: org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): Undefined column. columnName=UNMAPPED_FACTS.account
        at org.apache.phoenix.schema.PTableImpl.getColumnForColumnName(PTableImpl.java:818) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:478) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:208) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.TupleProjectionCompiler$ColumnRefVisitor.visit(TupleProjectionCompiler.java:194) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.TupleProjectionCompiler.createProjectedTable(TupleProjectionCompiler.java:109) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.QueryCompiler.compileSingleFlatQuery(QueryCompiler.java:528) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.QueryCompiler.compileSingleQuery(QueryCompiler.java:507) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.QueryCompiler.compileSelect(QueryCompiler.java:202) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.compile.QueryCompiler.compile(QueryCompiler.java:157) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:475) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.jdbc.PhoenixStatement$ExecutableSelectStatement.compilePlan(PhoenixStatement.java:441) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1648) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.jdbc.PhoenixStatement.compileQuery(PhoenixStatement.java:1641) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.jdbc.PhoenixStatement.optimizeQuery(PhoenixStatement.java:1635) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:260) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.phoenix.hive.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:131) ~[phoenix-4.11.0-HBase-1.3-hive.jar:?]
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:372) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:304) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:459) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:428) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:146) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2098) ~[hive-exec-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:479) ~[hive-service-2.1.1-amzn-0.jar:2.1.1-amzn-0]
        ... 25 more

Answer 1

[您只需要添加属性“ phoenix.column.mapping”，然后将每个配置单元表列映射到phoenix表，但映射中的每个左列都必须小写。

像魅力一样工作。