HttpClientFactory TaskCanceledException：无法从传输连接读取数据

Question

我创建了一个简单的控制台应用程序，可以使用new ASP.NET Core 2.1 HttpClientFactory从archive.org下载单个（PDF）文件。

对于该程序中使用的特定URL，我总是得到一个TaskCanceledException。如果尝试运行此代码，则可能会遇到相同的异常。但是，它适用于archive.org上的其他URL。使用wget从完全相同的URL（wget https://archive.org/download/1952-03_IF/1952-03_IF.pdf --output-document=IF.pdf）下载文件时下载成功。

但是，当我使用HttpClient进行操作时，出现以下异常。

我可能做错了什么？

这是简单的代码：

using System;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.DependencyInjection;
using System.IO;
using System.Diagnostics;

namespace test2
{
    public class Program
    {
        public static async Task Main(string[] args)
        {
            var serviceCollection = new ServiceCollection();
            serviceCollection.AddHttpClient("archive", c =>
            {
                c.BaseAddress = new Uri("https://archive.org/download/");
                c.DefaultRequestHeaders.Add("Accept", "application/pdf");
            })
            .AddTypedClient<ArchiveClient>();

            var services = serviceCollection.BuildServiceProvider();
            var archive = services.GetRequiredService<ArchiveClient>();
            await archive.Get();
        }

        private class ArchiveClient
        {
            public ArchiveClient(HttpClient httpClient)
            {
                HttpClient = httpClient;
            }

            public HttpClient HttpClient { get; }

            public async Task Get()
            {
                var request = new HttpRequestMessage(HttpMethod.Get, "1952-03_IF/1952-03_IF.pdf");
                var response = await HttpClient.SendAsync(request).ConfigureAwait(false);
                response.EnsureSuccessStatusCode();
                using (Stream contentStream = await response.Content.ReadAsStreamAsync(), 
                    fileStream = new FileStream("Worlds of IF 1952-03.pdf", FileMode.Create, FileAccess.Write, FileShare.None, 8192, true))
                {
                    var totalRead = 0L;
                    var totalReads = 0L;
                    var buffer = new byte[8192];
                    var isMoreToRead = true;

                    do
                    {
                        var read = await contentStream.ReadAsync(buffer, 0, buffer.Length);
                        if (read == 0)
                        {
                            isMoreToRead = false;
                        }
                        else
                        {
                            await fileStream.WriteAsync(buffer, 0, read);

                            totalRead += read;
                            totalReads += 1;

                            if (totalReads % 2000 == 0)
                            {
                                Console.WriteLine(string.Format("bytes downloaded: {0:n0}", totalRead));
                            }
                        }
                    }
                    while (isMoreToRead);
                }
            }
        }
    }
}

这是我得到的全部例外：

Unhandled Exception: System.Threading.Tasks.TaskCanceledException: The operation was canceled. 
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. 
---> System.Net.Sockets.SocketException: Operation canceled    
--- End of inner exception stack trace ---    
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)    
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)    
at System.Net.Security.SslStreamInternal.<FillBufferAsync>g__InternalFillBufferAsync|38_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)    
at System.Net.Security.SslStreamInternal.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)    
at System.Net.Http.HttpConnection.FillAsync()    
at System.Net.Http.HttpConnection.CopyToExactLengthAsync(Stream destination, UInt64 length, CancellationToken cancellationToken)    
at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)    
--- End of inner exception stack trace ---    
at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)    
at System.Net.Http.HttpConnection.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken) 
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)    at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)    
at test2.Program.ArchiveClient.Get() in /Users/Foo/Temp/test3/Program.cs:line 42    
at test2.Program.Main(String[] args) in /Users/Foo/Temp/test3/Program.cs:line 27    
at test2.Program.<Main>(String[] args)

Answer 1

在您的情况下，似乎是问题所在。我想尝试的另一件事是通过

HttpCompletionOption.ResponseHeadersRead

在SendAsync（）中作为第二个参数。发生的情况是，您的方法在读取标头后立即返回。该响应不再存储在MemoryStream缓冲区中，而是直接从套接字读取。这意味着您可以在流传输整个对象之前开始流传输。在性能方面，它要快得多，并且在您情况下，速度可能至关重要。

请记住要处理响应消息，否则连接将不会被释放。