在Spring Data MongoDB中返回Stream时指定游标选项？

Question

我正在使用Spring Data MongoDB（来自Spring Boot 1.5.2.RELEASE的spring-boot-starter-data-mongodb）和MongoDB 3.4.9，并定义了一个如下所示的存储库：

interface MyMongoDBRepository extends CrudRepository<MyDTO, String> {
    Stream<MyDTO> findAllByCategory(String category);
}

然后我有一个服务，MyService与此存储库交互：

@Service
class MyService {
    @Autowired
    MyMongoDBRepository repo;

    public void doStuff() {
        repo.findAllByCategory("category")
            .map(..)
            .filter(..)
            .forEach(..)
    }
}

数据库中存在大量数据，有时会发生此错误：

2018-01-01 18:16:56.631 ERROR 1 --- [ask-scheduler-6] o.s.integration.handler.LoggingHandler : org.springframework.dao.DataAccessResourceFailureException: 
Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server>; 
nested exception is com.mongodb.MongoCursorNotFoundException: 
Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server> 
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:77) 
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2135) 
at org.springframework.data.mongodb.core.MongoTemplate.access$1100(MongoTemplate.java:147) 
at org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2506)
at java.util.Iterator.forEachRemaining(Iterator.java:115) 
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) 
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) 
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) 
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) 
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) 
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) 
at com.mycompany.MyService.doStuff(MyService.java:108) 
at com.mycompany.AnotherService.doStuff(AnotherService.java:42) 
at sun.reflect.GeneratedMethodAccessor2026.invoke(Unknown Source) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) 
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) 
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81) 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:748) Caused by: com.mongodb.MongoCursorNotFoundException: Query failed with error code -5 and error message 'Cursor 73973161000 not found on server <mongodb-server>' on server <mongodb-server> 
at com.mongodb.operation.QueryHelper.translateCommandException(QueryHelper.java:27) 
at com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:213) 
at com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:103) 
at com.mongodb.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:46) 
at com.mongodb.DBCursor.hasNext(DBCursor.java:145) 
at org.springframework.data.mongodb.core.MongoTemplate$CloseableIterableCursorAdapter.hasNext(MongoTemplate.java:2504) ... 24 more

我在各个地方读到，当使用vanilla MongoDB Java客户端时，您可以将MongoDB游标配置为没有超时或设置批量大小以希望缓解这一点。

如果这是要走的路，那么从Spring Data MongoDB返回Stream时如何提供游标选项呢？

Answer 1

从Spring Data MongoDB返回Stream时，您不需要提供游标选项。此异常的可能原因是您的服务如何从Mongo读取数据。可能的原因：

1. 您正在跨多个线程共享一个游标
2. 一次要求您提供太多元素
3. Mongo服务器之前的负载均衡器

请参阅this Jira主题对某些想法的评论以及适用于您的应用程序的方向。

Answer 2

关于你提到的两个选项。

1. 批量大小，您无法使用Repository类设置批量大小。你可以用MongoTemplate来做。像这样的东西

   final DBCursor cursor = mongoTemplate
           .getCollection(collectionName)
           .find(queryBuilder.get(), projection)
           .batchSize(readBatchSize);
      while (cursor.hasNext()) {
          ......
          ......
       }

但是要使用MongoTemplate，您需要创建一个自定义存储库。

2. 关于游标超时。你可以做这样的事情

   @Configuration  
   public class MongoDbSettings {
   
       @Bean
       public MongoClientOptions setmongoOptions() {
           return MongoClientOptions.builder().socketTimeout(5000).build();
       }
    }

您可以为Mongo设置许多其他选项（heartbeat，connectiontimeout）。您可以在application.properties文件中设置这些属性，然后在上面的类中使用@Value绑定它并设置（而不是硬编码）。 不幸的是，spring-boot没有提供任何方法来在application.properties文件中指定它们