Ignite服务器节点上的内存不足

Question

我已经设置了一个ignite 2.3服务器节点以及32个客户端节点。运行多个查询后，我在服务器节点日志中观察到内存不足错误。

服务器配置：

1. 配置4 GB java max堆内存。
2. Ignite Persistence已禁用
3. 使用默认数据区域。
4. 使用Spring Data在ignite节点上应用查询。

捕获点燃服务器节点的内存快照。

- Node [id= =44:33:12.948]
    ^-- H/N/C [hosts=32, nodes=32, CPU =39]
    ^-- CPU [cur=3.7%, avg=0.23%, G C =0%]
    ^-- Page Memory [pages=303325]
    ^-- Heap [used=2404 MB, free=36.21%, comm=3769 MB]
    ^-- Non heap [used=78 MB, free=-1%, comm=80 MB]
    ^-- Public thread pool [active=0, idle=0, q Size =0]
    ^-- System thread pool [active=0, idle=6, q Size=0]
    ^-- Outbound messages queue [size=0]

堆转储日志分析：

query-#8779
  at java.nio.Bits$1.newDirectByteBuffer(JILjava/lang/Object;)Ljava/nio/ByteBuffer; (Bits.java:758)
  at org.apache.ignite.internal.util.GridUnsafe.wrapPointer(JI)Ljava/nio/ByteBuffer; (GridUnsafe.java:113)
  at org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl.pageBuffer(J)Ljava/nio/ByteBuffer; (PageMemoryNoStoreImpl.java:253)
  at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(Lorg/apache/ignite/internal/processors/cache/CacheGroupContext;Lorg/apache/ignite/internal/processors/cache/GridCacheSharedContext;Lorg/apache/ignite/internal/pagemem/PageMemory;Lorg/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter$RowData;)V (CacheDataRowAdapter.java:167)
  at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(Lorg/apache/ignite/internal/processors/cache/CacheGroupContext;Lorg/apache/ignite/internal/processors/cache/persistence/CacheDataRowAdapter$RowData;)V (CacheDataRowAdapter.java:102)
  at org.apache.ignite.internal.processors.query.h2.database.H2RowFactory.getRow(J)Lorg/apache/ignite/internal/processors/query/h2/opt/GridH2Row; (H2RowFactory.java:62)
  at org.apache.ignite.internal.processors.query.h2.database.io.H2ExtrasLeafIO.getLookupRow(Lorg/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree;JI)Lorg/h2/result/SearchRow; (H2ExtrasLeafIO.java:126)
  at org.apache.ignite.internal.processors.query.h2.database.io.H2ExtrasLeafIO.getLookupRow(Lorg/apache/ignite/internal/processors/cache/persistence/tree/BPlusTree;JI)Ljava/lang/Object; (H2ExtrasLeafIO.java:36)
  at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/BPlusIO;JILjava/lang/Object;)Lorg/apache/ignite/internal/processors/query/h2/opt/GridH2Row; (H2Tree.java:123)
  at org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(Lorg/apache/ignite/internal/processors/cache/persistence/tree/io/BPlusIO;JILjava/lang/Object;)Ljava/lang/Object; (H2Tree.java:40)
  at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer(JLorg/apache/ignite/internal/processors/cache/persistence/tree/io/BPlusIO;II)Z (BPlusTree.java:4548)
  at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.nextPage()Z (BPlusTree.java:4641)
  at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next()Z (BPlusTree.java:4570)
  at org.apache.ignite.internal.processors.query.h2.H2Cursor.next()Z (H2Cursor.java:78)
  at org.h2.index.IndexCursor.next()Z (IndexCursor.java:305)
  at org.h2.table.TableFilter.next()Z (TableFilter.java:499)
  at org.h2.command.dml.Select$LazyResultQueryFlat.fetchNextRow()[Lorg/h2/value/Value; (Select.java:1452)
  at org.h2.result.LazyResult.hasNext()Z (LazyResult.java:79)
  at org.h2.result.LazyResult.next()Z (LazyResult.java:59)
  at org.h2.command.dml.Select.queryFlat(ILorg/h2/result/ResultTarget;J)Lorg/h2/result/LazyResult; (Select.java:519)
  at org.h2.command.dml.Select.queryWithoutCache(ILorg/h2/result/ResultTarget;)Lorg/h2/result/ResultInterface; (Select.java:625)
  at org.h2.command.dml.Query.queryWithoutCacheLazyCheck(ILorg/h2/result/ResultTarget;)Lorg/h2/result/ResultInterface; (Query.java:114)
  at org.h2.command.dml.Query.query(ILorg/h2/result/ResultTarget;)Lorg/h2/result/ResultInterface; (Query.java:352)
  at org.h2.command.dml.Query.query(I)Lorg/h2/result/ResultInterface; (Query.java:333)
  at org.h2.command.CommandContainer.query(I)Lorg/h2/result/ResultInterface; (CommandContainer.java:113)
  at org.h2.command.Command.executeQuery(IZ)Lorg/h2/result/ResultInterface; (Command.java:201)
  at org.h2.jdbc.JdbcPreparedStatement.executeQuery()Ljava/sql/ResultSet; (JdbcPreparedStatement.java:111)
  at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQuery(Ljava/sql/Connection;Ljava/sql/PreparedStatement;ILorg/apache/ignite/internal/processors/query/GridQueryCancel;)Ljava/sql/ResultSet; (IgniteH2Indexing.java:961)
  at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(Ljava/sql/PreparedStatement;Ljava/sql/Connection;Ljava/lang/String;Ljava/util/Collection;ILorg/apache/ignite/internal/processors/query/GridQueryCancel;)Ljava/sql/ResultSet; (IgniteH2Indexing.java:1027)
  at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSqlQueryWithTimer(Ljava/sql/Connection;Ljava/lang/String;Ljava/util/Collection;ZILorg/apache/ignite/internal/processors/query/GridQueryCancel;)Ljava/sql/ResultSet; (IgniteH2Indexing.java:1006)

Answer 1

您是否有可能尝试在没有SELECT *子句或具有巨大结果集的类似请求的情况下执行WHERE？结果集将保留在堆上，这将在提供此类请求时导致OOM。

在SQL查询中使用LIMIT子句，或在lazy=true / Connection中使用SqlFieldsQuery。