janusGraphManagement.getOrCreatePropertyKey() 导致无法读取时间戳消息

问题描述 投票:0回答:0

问题

为什么一个空的 JanusGraph 不能

read messages for timestamp

JanusGraph 为
janusGraphManagement.getOrCreatePropertyKey()
指的是什么消息?
这个警告是如何解决的,为什么添加数据时会产生这么多警告?

[WARN] [o.j.d.l.k.KCVSLog.pool-5-thread-1] ::    Could not read messages for timestamp [2023-05-17T12:50:45.282434100Z] (this read will be retried)
java.lang.IllegalArgumentException: Timestamp overflow detected: 2023-05-17T12:50:45.282434100Z
        logger.info("g.V().count().next():\t" + g.V().count().next());
        PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey("_id");

繁殖

步骤

  1. 创建并启动一个 Cassandra [Docker 容器]
    docker run --name jg-cassandra -d -e CASSANDRA_START_RPC=true -p 9160:9160 -p 9042:9042 -p 7199:7199 -p 7001:7001 -p 7000:7000 cassandra:3.11
    
  2. 将大型(或至少非平凡的)数据集加载到 JanusGraph 中
    1. “中型”数据集样本是ICIJ Offshore Dataleaks
    2. 中等意味着比某人可以硬编码不同和“随机”数据更大的数据集
  3. 创建并运行这个 Java + Maven 项目(代码如下)
  4. 结果
    1. 预期:JanusGraph-Cassandra 处理读取消息
    2. 实际上:当这些 2 行存在时,JanusGraph-Cassandra 将抛出消息,而不管设置如何
      [WARN] [o.j.d.l.k.KCVSLog.pool-5-thread-1] ::    Could not read messages for timestamp [2023-05-17T12:50:45.282434100Z] (this read will be retried)
      java.lang.IllegalArgumentException: Timestamp overflow detected: 2023-05-17T12:50:45.282434100Z
      

试过

  1. 添加数据以查看它是否是 null 或零长度问题
    这只会让错误重复得更多,等待完成的时间更长
    g.V()
    现在问题永久存在
  2. 创建一个新的 Docker 容器
    问题仍然存在
  3. 使用不同的
    g.V()
    执行
    g.V.drop().iterate()

    Issue 还是一样的回复
  4. getOrCreatePropertyKey()
    发生时的顺序更改为
    g.V()
    发生时
    getOrCreatePropertyKey()
    被调用
  5. 时问题仍然发生
  6. 评论
    getOrCreatePropertyKey()
    并离开
    g.V().count().next()

    问题消失
  7. 评论
    g.V().count().next()
    并离开
    getOrCreatePropertyKey()

    问题消失
  8. 评论每个
    getOrCreatePropertyKey()
    g.V()
    back-in
    问题回来了
  9. 改变时间戳的处理方式
    set(TIMESTAMP_PROVIDER.toStringWithoutRoot(), "NANO")

    问题没有改变
  10. 切换到 Java-11 [corretto-11 11.0.19]
    问题仍然存在

日志

2023-05-17 07:50:43,822 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::   Contact point localhost:29042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-17 07:50:43,930 [INFO] [c.d.o.d.i.c.DefaultMavenCoordinates.main] ::     DataStax Java driver for Apache Cassandra(R) (com.datastax.oss:java-driver-core) version 4.15.0
2023-05-17 07:50:44,444 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::   Using native clock for microsecond precision
2023-05-17 07:50:44,725 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::     [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/[0:0:0:0:0:0:0:1]:29042, hostId=null, hashCode=5aad4393)=null; please provide the correct local DC, or check your contact points
2023-05-17 07:50:44,987 [INFO] [o.j.g.i.UniqueInstanceIdRetriever.main] ::   Generated unique-instance-id=c0a8564916248-rmt-lap-win201
2023-05-17 07:50:45,011 [INFO] [c.d.o.d.i.c.ContactPoints.main] ::   Contact point localhost:29042 resolves to multiple addresses, will use them all ([localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1])
2023-05-17 07:50:45,045 [INFO] [c.d.o.d.i.c.t.Clock.JanusGraph Session-admin-0] ::   Using native clock for microsecond precision
2023-05-17 07:50:45,086 [WARN] [c.d.o.d.i.c.l.h.OptionalLocalDcHelper.JanusGraph Session-admin-0] ::     [JanusGraph Session|default] You specified datacenter1 as the local DC, but some contact points are from a different DC: Node(endPoint=localhost/[0:0:0:0:0:0:0:1]:29042, hostId=null, hashCode=83832b8)=null; please provide the correct local DC, or check your contact points
2023-05-17 07:50:45,112 [INFO] [o.j.d.c.ExecutorServiceBuilder.main] ::  Initiated fixed thread pool of size 40
2023-05-17 07:50:45,254 [INFO] [o.j.g.d.StandardJanusGraph.main] ::  Gremlin script evaluation is disabled
2023-05-17 07:50:45,282 [INFO] [o.j.d.l.k.KCVSLog.main] ::   Loaded unidentified ReadMarker start time 2023-05-17T12:50:45.282434100Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@48b4a043
2023-05-17 07:50:45,346 [WARN] [o.j.g.t.StandardJanusGraphTx.main] ::    Query requires iterating over all vertices [[]]. For better performance, use indexes
2023-05-17 07:50:45,359 [INFO] [Main.main] ::    g.V().count().next():  0
2023-05-17 07:50:45,393 [WARN] [o.j.d.l.k.KCVSLog.pool-5-thread-1] ::    Could not read messages for timestamp [2023-05-17T12:50:45.282434100Z] (this read will be retried)
java.lang.IllegalArgumentException: Timestamp overflow detected: 2023-05-17T12:50:45.282434100Z
    at org.janusgraph.diskstorage.log.kcvs.KCVSLog.getTimeSlice(KCVSLog.java:393) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.log.kcvs.KCVSLog.access$1500(KCVSLog.java:115) ~[janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:743) [janusgraph-core-1.0.0-20230504-014643.988c094.jar:?]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
    at java.lang.Thread.run(Thread.java:833) [?:?]

Process finished with exit code 0

代码

public class Test {
    private static final Logger logger = LogManager.getLogger(Main.class);

    public static void main(String[] args) throws InterruptedException, ExecutionException {
        JanusGraph janusGraph = JanusGraphFactory.build()
                .set("storage.backend", "cql")
                .set("storage.hostname", "localhost:29042")
//                .set(TIMESTAMP_PROVIDER.toStringWithoutRoot(), "NANO")
                .open();
        JanusGraphManagement janusGraphManagement = janusGraph.openManagement();
        GraphTraversalSource g = janusGraph.traversal();
        logger.info("g.V().count().next():\t" + g.V().count().next());
//        g.V().limit(10000).drop().iterate();
        PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey("_id");
//        GraphTraversalSource g = janusGraph.traversal();
//        logger.info("g.V().count().next():\t" + g.V().count().next());
        janusGraph.close();
    }

资源

  1. Maven 3.8.1
  2. Java 11.0.19 (corretto-11)
  3. JanusGraph 1.0.0-20230504-014643.988c094
  4. Windows 10
  5. 码头工人
  6. Cassandra:latest [容器]
  7. 哈佛肺癌数据集
gremlin traversal cassandra-3.0 docker-container janusgraph
© www.soinside.com 2019 - 2024. All rights reserved.