在ASP.NET Core 中使用HttpClient 或WebRequest 来远程异步逐行读取文件内容更好吗？

Question

我计划使用 https://github.com/Dasync/AsyncEnumerable (since there is not yet Async Streams [C# 8 maybe]: https://github.com/dotnet/csharplang/blob/master/proposals/async-streams.md):

异步逐行读取远程文件

public static class StringExtensions
{
    public static AsyncEnumerable<string> ReadLinesAsyncViaHttpClient(this string uri)
    {
        return new AsyncEnumerable<string>(async yield =>
        {
            using (var httpClient = new HttpClient())
            {
                using (var responseStream = await httpClient.GetStreamAsync(uri))
                {
                    using (var streamReader = new StreamReader(responseStream))
                    {
                        while(true)
                        {
                            var line = await streamReader.ReadLineAsync();

                            if (line != null)
                            {
                                await yield.ReturnAsync(line);
                            }
                            else
                            {
                                return;
                            }
                        } 
                    }
                }
            }
        });
    }
    public static AsyncEnumerable<string> ReadLinesAsyncViaWebRequest(this string uri)
    {
        return new AsyncEnumerable<string>(async yield =>
        {
            var request = WebRequest.Create(uri);
            using (var response = request.GetResponse())
            {
                using (var responseStream = response.GetResponseStream())
                {
                    using (var streamReader = new StreamReader(responseStream))
                    {
                        while(true)
                        {
                            var line = await streamReader.ReadLineAsync();

                            if (line != null)
                            {
                                await yield.ReturnAsync(line);
                            }
                            else
                            {
                                return;
                            }
                        } 
                    }
                }
            }
        });
    }
}

似乎它们都运行在如下所示的简单控制台应用程序中都很好：

public class Program
{
    public static async Task Main(string[] args)
    {
        // Or any other remote file
        const string url = @"https://gist.githubusercontent.com/dgrtwo/a30d99baa9b7bfc9f2440b355ddd1f75/raw/700ab5bb0b5f8f5a14377f5103dbe921d4238216/by_tag_year.csv";

        await url.ReadLinesAsyncViaWebRequest().ForEachAsync(line =>
        {
            Console.WriteLine(line, Color.GreenYellow);
        });
        await url.ReadLinesAsyncViaHttpClient().ForEachAsync(line =>
        {
            Console.WriteLine(line, Color.Purple);
        });
    }
}

...但如果将其用作 ASP.NET 核心 WebAPI 的一部分来处理行然后使用 PushStreamContent 推送它们，我会有些担心：

我们的想法是拥有一个利用 async / await 的数据管道，以便使用的线程数尽可能少，同时避免内存增加（它利用了 AsyncEnumerable 的类似可枚举的特性。

我读了几篇文章，但似乎都是非 .NET Core 版本，我真的不知道关于我想要实现的目标是否存在一些潜在的性能问题/注意事项？

http://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/

"business" 案例的一个例子是：

using System;
using System.Collections.Async;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;

namespace WebApplicationTest.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class DumbValuesController : ControllerBase
    {
        private static readonly Random Random = new Random();

        // GET api/values
        [HttpGet]
        public async Task<IActionResult> DumbGetAsync([FromQuery] string fileUri)
        {
            using (var streamWriter = new StreamWriter(HttpContext.Response.Body))
            {
                await fileUri.ReadLinesAsyncViaHttpClient().ForEachAsync(async line =>
                {
                    // Some dumb process on each (maybe big line)
                    line += Random.Next(0, 100 + 1);
                    await streamWriter.WriteLineAsync(line);
                });
            }

            return Ok();
        }
    }
}

Answer 1

我们可以访问 .NET Core 的源代码。所以你可以看看。

两者的底层实现最终都使用 HttpClientHandler（class 的实现分为 4 个文件）。

这两个HttpClient and HttpWebRequest (which WebRequest uses).

的源码可以看出

所以我怀疑您不会注意到两者在性能上有任何差异。

HttpClient 是最新编写的，因此鼓励使用它。由于您链接到的文章中提到的原因：http://www.diogonunes.com/blog/webclient-vs-httpclient-vs-httpwebrequest/

Answer 2

在 .Net Core 6.0 的最新版本中，WebRequest 将被声明为已弃用。 Microsoft 建议改用 HttpClient

https://docs.microsoft.com/en-us/dotnet/core/compatibility/networking/6.0/webrequest-deprecated

在ASP.NET Core 中使用HttpClient 或WebRequest 来远程异步逐行读取文件内容更好吗？

Is it better to use HttpClient or WebRequest in ASP.NET Core to read the content of a file line by line asynchronously remotely?

c#

webrequest

dotnet-httpclient

.net-core

asp.net-core