我在使用 Apache Camel Http 组件从公共链接下载/流式传输文件到外部系统 (SharePoint) 时遇到内存不足错误。看来该组件内部正在将所有文件一次读入内存。我们可以在不将整个流加载到内存的情况下做到这一点吗？路线很简单。
/**
* Submit file to upload
*/
from("direct:fetch-content-data-and-upload")
.log("** Fetching file content from public link")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.toD("${header.CamelHttpUri}")
.process(this.submitFileToUpload)
.end();
处理器内部使用Graph Client SDK提交Input Stream以分块上传。错误在到达处理器之前被抛出。
Aug 3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onIdleExpired(HttpConnectionOverHTTP.java:160)
Aug 3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.ssl.SslConnection.onIdleExpired(SslConnection.java:391)
Aug 3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:402)
Aug 3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:171)
Aug 3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Aug 3 07:38:37 ip-172-31-9-176 web: ... 3 common frames omitted
Aug 3 07:38:37 ip-172-31-9-176 web: Caused by: java.lang.OutOfMemoryError: Java heap space
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.Arrays.copyOf(Arrays.java:3537) ~[na:na]
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100) ~[na:na]
Aug 3 07:38:37 ip-172-31-9-176 web: at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:130) ~[na:na]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.converter.stream.CachedOutputStream.write(CachedOutputStream.java:101) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:206) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:161) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:156) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:152) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.doExtractResponseBodyAsStream(HttpProducer.java:547) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.extractResponseBody(HttpProducer.java:532) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.populateResponse(HttpProducer.java:332) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:291) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor.lambda$process$0(SendDynamicProcessor.java:197) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor$$Lambda$1001/0x000000d8016ec350.doInAsyncProducer(Unknown Source) ~[na:na]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:327) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:182) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:477) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:181) ~[camel-base-engine-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.impl.engine.DefaultReactiveExecutor.schedule(DefaultReactiveExecutor.java:54) ~[camel-base-engine-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.lambda$redeliver$2(RedeliveryErrorHandler.java:914) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug 3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask$$Lambda$994/0x000000d8016c1360.done(Unknown Source) ~[na:na] ```