Trino 无法连接到 Nessie

问题描述 投票:0回答:1

我有一个运行 Spark/Iceberg/MinIO 的应用程序以及 Iceberg 的 Nessie 目录,一切正常(Iceberg 数据已正确加载到 MinIO 和 Nessie)。

现在我尝试连接 Trino,但对“http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL”执行 GET 请求时失败(当我从浏览器执行时,此请求有效,或者失眠)。

在 Trino GUI 中,这是由“java.net.ConnectException:未定义”引起的。

我错过了什么?

我正在使用 Trino 官方 Helm Chart 0.18.0(运行 Trino 版本 432)并在 Nessie 0.77.1(当前最新)和 0.71.0(与 Trino 432 pom.xml 中的 Nessie 版本匹配:https:/ /github.com/trinodb/trino/blob/432/plugin/trino-iceberg/pom.xml

我的 Trino 连接器属性:

connector.name=iceberg
iceberg.catalog.type=nessie
iceberg.nessie-catalog.uri=http://localhost:19120/api/v1
iceberg.nessie-catalog.default-warehouse-dir=s3a://data/test
hive.s3.aws-access-key=VUtRVIf0hg7szCp3k0Pz
hive.s3.aws-secret-key=lHzYClEjh2AH5mEfRdPS720pMl3UZl7riR3uL4pL
hive.s3.endpoint=http://127.0.0.1:9000
hive.s3.path-style-access=true
hive.s3.ssl.enabled=false

我的 Spark 会话配置(Java):

SparkSession.builder()
    .appName("Test App")
    .master("local")
    .config("spark.ui.enabled", "false")
    .config("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem")
    .config("fs.s3a.endpoint", "http://127.0.0.1:9000")
    .config("fs.s3a.access.key", VUtRVIf0hg7szCp3k0Pz)
    .config("fs.s3a.secret.key", lHzYClEjh2AH5mEfRdPS720pMl3UZl7riR3uL4pL)
    .config("fs.s3a.path.style.access", "true")
    .config("spark.sql.warehouse.dir", WAREHOUSE_PATH)
    .config("spark.jars.packages", "org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:1.3.0,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.5_2.13:0.77.1")
    .config("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions")
    .config("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog")
    .config("spark.sql.catalog.nessie.catalog-impl", "org.apache.iceberg.nessie.NessieCatalog")
    .config("spark.sql.catalog.nessie.authentication.type", "NONE")
    .config("spark.sql.catalog.nessie.uri", "http://localhost:19120/api/v1")
    .config("spark.sql.catalog.nessie.ref", "main")
    .config("spark.sql.defaultCatalog", "nessie")
    .config("spark.sql.catalog.nessie.warehouse", "s3a://data/test")
    .getOrCreate();

来自 Trino GUI 的整个堆栈跟踪:

io.trino.spi.TrinoException: Error listing schemas for catalog nessie: Failed to execute GET request against 'http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL'.
    at io.trino.metadata.MetadataListing.handleListingException(MetadataListing.java:349)
    at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:97)
    at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:88)
    at io.trino.connector.informationschema.InformationSchemaPageSource.addSchemataRecords(InformationSchemaPageSource.java:322)
    at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:226)
    at io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:184)
    at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:266)
    at io.trino.operator.Driver.processInternal(Driver.java:395)
    at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
    at io.trino.operator.Driver.tryWithLock(Driver.java:694)
    at io.trino.operator.Driver.process(Driver.java:290)
    at io.trino.operator.Driver.processForDuration(Driver.java:261)
    at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
    at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
    at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
    at io.trino.$gen.Trino_432____20240219_102654_2.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.projectnessie.client.http.HttpClientException: Failed to execute GET request against 'http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL'.
    at org.projectnessie.client.http.impl.jdk11.JavaRequest.executeRequest(JavaRequest.java:128)
    at org.projectnessie.client.http.HttpRequest.get(HttpRequest.java:80)
    at org.projectnessie.client.rest.v1.RestV1TreeClient.getReferenceByName(RestV1TreeClient.java:83)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at org.projectnessie.client.rest.v1.RestV1Client$ExceptionRewriter.invoke(RestV1Client.java:78)
    at jdk.proxy4/jdk.proxy4.$Proxy236.getReferenceByName(Unknown Source)
    at org.projectnessie.client.rest.v1.HttpGetReference.get(HttpGetReference.java:34)
    at org.apache.iceberg.nessie.NessieIcebergClient.loadReference(NessieIcebergClient.java:114)
    at org.apache.iceberg.nessie.NessieIcebergClient.lambda$new$0(NessieIcebergClient.java:81)
    at org.apache.iceberg.relocated.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:181)
    at org.apache.iceberg.nessie.NessieIcebergClient.getRef(NessieIcebergClient.java:89)
    at org.apache.iceberg.nessie.NessieIcebergClient.withReference(NessieIcebergClient.java:501)
    at org.apache.iceberg.nessie.NessieIcebergClient.listNamespaces(NessieIcebergClient.java:207)
    at io.trino.plugin.iceberg.catalog.nessie.TrinoNessieCatalog.listNamespaces(TrinoNessieCatalog.java:104)
    at io.trino.plugin.iceberg.IcebergMetadata.listSchemaNames(IcebergMetadata.java:355)
    at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.listSchemaNames(ClassLoaderSafeConnectorMetadata.java:197)
    at io.trino.tracing.TracingConnectorMetadata.listSchemaNames(TracingConnectorMetadata.java:131)
    at io.trino.metadata.MetadataManager.listSchemaNames(MetadataManager.java:254)
    at io.trino.tracing.TracingMetadata.listSchemaNames(TracingMetadata.java:168)
    at io.trino.metadata.MetadataListing.doListSchemas(MetadataListing.java:103)
    at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:94)
    ... 17 more
Caused by: java.net.ConnectException: undefined
    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:951)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
    at org.projectnessie.client.http.impl.jdk11.JavaHttpClient.lambda$newRequest$0(JavaHttpClient.java:68)
    at org.projectnessie.client.http.impl.jdk11.JavaRequest.executeRequest(JavaRequest.java:110)
    ... 38 more
Caused by: java.net.ConnectException: undefined
    at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1028)
    at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:227)
    at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(PlainHttpConnection.java:280)
    at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$2(PlainHttpConnection.java:238)
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
    ... 3 more
Caused by: java.nio.channels.ClosedChannelException: undefined
    at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:202)
    at java.base/sun.nio.ch.SocketChannelImpl.beginConnect(SocketChannelImpl.java:786)
    at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:874)
    at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:210)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
    at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:212)
    ... 9 more
apache-spark trino apache-iceberg nessie
1个回答
0
投票

确保可以从 Trino 服务器内访问 Nessie 服务器。我有同样的错误并意识到我设置了

iceberg.nessie-catalog.uri=http://localhost:19120/api/v1

但是,我在 Docker 容器内运行 Trino,因此 localhost 引用了自身。设置

--network=host
解决了这个问题,但您也可以在同一网络下配置两个容器(如果您也在容器中运行 Nessie)。

© www.soinside.com 2019 - 2024. All rights reserved.