如何修复org.neo4j.internal.unsafe.NativeMemoryAllocationRefusedError？

Question

很高兴获得有关以下问题的任何帮助。 我需要用文档的文字创建一个 neo4j 图表。对于 1000 个文档，它总是工作正常，但对于 3000 个文档，我会遇到此内存错误。我已经使用 -Xmx4096m 增加了堆大小，但这并没有解决问题。我将非常感谢这个问题的任何答案。

Exception in thread "main" org.neo4j.internal.unsafe.NativeMemoryAllocationRefusedError: Failed to allocate 524288 bytes. So far 36235008 bytes have already been successfully allocated. The system currently has 16930299904 bytes total physical memory, 57805078528 bytes committed virtual memory, and 570298368 bytes free physical memory. Relevant system properties: "java.vm.name" = "OpenJDK 64-Bit Server VM", "java.vm.vendor" = "OpenLogic", "os.arch" = "amd64".
    at org.neo4j.internal.unsafe.UnsafeUtil.allocateMemory(UnsafeUtil.java:444)
    at org.neo4j.io.mem.GrabAllocator$Grab.<init>(GrabAllocator.java:90)
    at org.neo4j.io.mem.GrabAllocator$Grabs.allocateAligned(GrabAllocator.java:244)
    at org.neo4j.io.mem.GrabAllocator.allocateAligned(GrabAllocator.java:70)
    at org.neo4j.io.pagecache.impl.muninn.PageList.initBuffer(PageList.java:299)
    at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pageFault(MuninnPageCursor.java:419)
    at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.uncommonPin(MuninnPageCursor.java:369)
    at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.pin(MuninnPageCursor.java:337)
    at org.neo4j.io.pagecache.impl.muninn.MuninnReadPageCursor.next(MuninnReadPageCursor.java:63)
    at org.neo4j.io.pagecache.impl.muninn.MuninnPageCursor.next(MuninnPageCursor.java:160)
    at org.neo4j.kernel.impl.store.CommonAbstractStore.readIntoRecord(CommonAbstractStore.java:885)
    at org.neo4j.kernel.impl.store.CommonAbstractStore.getRecordByCursor(CommonAbstractStore.java:869)
    at org.neo4j.internal.recordstorage.RecordPropertyCursor.property(RecordPropertyCursor.java:439)
    at org.neo4j.internal.recordstorage.RecordPropertyCursor.next(RecordPropertyCursor.java:160)
    at org.neo4j.kernel.impl.newapi.DefaultPropertyCursor.next(DefaultPropertyCursor.java:163)
    at org.neo4j.kernel.impl.newapi.DefaultPropertyCursor.seekProperty(DefaultPropertyCursor.java:235)
    at org.neo4j.kernel.impl.core.NodeEntity.getProperty(NodeEntity.java:439)
    at core.Cooccs.addSentenceToCooccsDB(Cooccs.java:661)
    at core.Cooccs.<init>(Cooccs.java:577)
    at core.TextProcessing.createDB(TextProcessing.java:386)
    at core.TextProcessing.main(TextProcessing.java:1085)

Answer 1

您可以使用批处理来减少单个事务的内存需求。

您可以使用 CALL 子查询或 APOC 批量事务处理程序进行批处理。