将数据从 Hive 摄取到 GeoMesa Accumulo

问题描述 投票:0回答:1

我正在尝试使用 JDBC 转换器将数据从 Hive 提取到 GeoMesa Accumulo 中。这是我的 JDBC 转换器的配置:

geomesa.converters.hive  = {    
  type = "jdbc",   
  connection = "jdbc:hive2://host:10000/default",
  id-field = "$1",
  fields = [
    { name = "my_id", type = "string", transform = "$1" },
    { name = "shape", type = "geometry", transform = "geometry($2)" }
  ]
}

要摄取数据,我使用以下命令:

echo "SELECT * FROM table" | geomesa-accumulo ingest -u user -p pass -c catalog -s hive.sft -f feature -C hive.conf

尽管已将 Hive JDBC 驱动程序复制到 GeoMesa 库,但在运行上述命令时遇到以下错误:

2023-09-13 10:51:59,640 WARN  [org.locationtech.geomesa.convert2.package$ErrorHandlingIterator] Failed parsing input: 
java.sql.SQLFeatureNotSupportedException: Method not supported
    at org.apache.hive.jdbc.HiveConnection.prepareCall(HiveConnection.java:1215)
    at org.locationtech.geomesa.convert.jdbc.JdbcConverter$StatementIterator.hasNext(JdbcConverter.scala:86)
    at org.locationtech.geomesa.convert2.package$ErrorHandlingIterator.liftedTree1$1(package.scala:84)
    at org.locationtech.geomesa.convert2.package$ErrorHandlingIterator.hasNext(package.scala:83)
    at org.locationtech.geomesa.convert.jdbc.JdbcConverter$ResultSetIterator.hasNext(JdbcConverter.scala:119)
    at org.locationtech.geomesa.utils.collection.CloseableIterator$FlatMapCloseableIterator.hasNext(CloseableIterator.scala:140)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at org.locationtech.geomesa.utils.collection.CloseableIterator$FlatMapCloseableIterator.foreach(CloseableIterator.scala:132)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$7(LocalConverterIngest.scala:179)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$7$adapted(LocalConverterIngest.scala:173)
    at org.locationtech.geomesa.utils.io.CloseablePool$CommonsPoolPool.borrow(CloseablePool.scala:68)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$6(LocalConverterIngest.scala:173)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$6$adapted(LocalConverterIngest.scala:172)
    at org.locationtech.geomesa.utils.io.package$WithClose$.apply(package.scala:64)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$4(LocalConverterIngest.scala:172)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$4$adapted(LocalConverterIngest.scala:168)
    at scala.collection.Iterator.foreach(Iterator.scala:943)
    at scala.collection.Iterator.foreach$(Iterator.scala:943)
    at org.locationtech.geomesa.utils.collection.CloseableIterator$CloseableSingleIterator.foreach(CloseableIterator.scala:85)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$3(LocalConverterIngest.scala:168)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$3$adapted(LocalConverterIngest.scala:167)
    at org.locationtech.geomesa.utils.io.package$WithClose$.apply(package.scala:64)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$2(LocalConverterIngest.scala:167)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.$anonfun$run$2$adapted(LocalConverterIngest.scala:166)
    at org.locationtech.geomesa.utils.io.CloseablePool$CommonsPoolPool.borrow(CloseablePool.scala:68)
    at org.locationtech.geomesa.tools.ingest.LocalConverterIngest$LocalIngestWorker.run(LocalConverterIngest.scala:166)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)

有人有解决这个问题的见解吗?

hadoop jdbc hive accumulo geomesa
1个回答
0
投票

这似乎是 GeoMesa 中的一个错误。它应该调用

prepareStatement
而不是
prepareCall
,尽管两者似乎都可以与其他数据库(例如 Postgresql)一起使用。我已在此处开立了一张票来跟踪该问题。

如果可能,您可以尝试更改此行以使用

prepareStatement
从源代码构建,然后看看是否可以解决您的问题。

© www.soinside.com 2019 - 2024. All rights reserved.