Parallel.ForEach和任务请求处理请求失败问题

Question

我正在对应用程序进行一些性能优化，因此开始进行并行调用。但是在数据处理中看到了很多差异，它完全不一致哪个请求将被拒绝（失败）。下面是示例代码及其在 4 轮测试中的结果。尝试进行 50 次并行 API 调用，但每次都有一些请求失败。每次您可以通过 Parallel.ForEach 和 Task

丢弃一些请求

50个样本的并行处理分析API调用

Round-1
Time taken by Parallel.Foreach :00:00:00.2732286  -  Count:50
Time taken by Task             :00:00:00.0025059  -  Count:48
Time taken by Foreach          :00:00:08.3770130  -  Count:50

Round-2
Time taken by Parallel.Foreach :00:00:00.1271151  -  Count:46
Time taken by Task             :00:00:00.0005574  -  Count:50
Time taken by Foreach          :00:00:05.3288707  -  Count:50

Round-3
Time taken by Parallel.Foreach :00:00:00.1224027  -  Count:49
Time taken by Task             :00:00:00.0003799  -  Count:50
Time taken by Foreach          :00:00:05.2718811  -  Count:50

Round-4
Time taken by Parallel.Foreach :00:00:00.1295570  -  Count:49
Time taken by Task             :00:00:00.0004238  -  Count:48
Time taken by Foreach          :00:00:05.2395539  -  Count:50

C#示例代码

async static Task Main(string[] args)
        {
            HttpClient httpClient = new HttpClient();

            var res1 = new List<HttpResponseMessage>();
            var res2 = new List<HttpResponseMessage>();
            var res3 = new List<HttpResponseMessage>();

            var countlist = new List<int>();

            for (int i = 1; i <= 50; i++)
            {
                countlist.Add(i);
            }

            // Parallel foreach
            Stopwatch stopwatch1 = Stopwatch.StartNew();
            Parallel.ForEach(countlist, async (item) =>
             {
                 var data = await httpClient.GetAsync("https://jsonplaceholder.typicode.com/todos/1");
                 res1.Add(data);
             });
            stopwatch1.Stop();

            // Task parallel
            Stopwatch stopwatch2 = Stopwatch.StartNew();

            var listtask = new List<Task>();
            foreach (var item in countlist)
            {
                listtask.Add(Task.Run(async () =>
                {
                    var data = await httpClient.GetAsync("https://jsonplaceholder.typicode.com/todos/1");
                    res2.Add(data);
                }));
            }
            stopwatch2.Stop();


            // Normal foreach
            Stopwatch stopwatch3 = Stopwatch.StartNew();
            foreach (var item in countlist)
            {
                var data = await httpClient.GetAsync("https://jsonplaceholder.typicode.com/todos/1");
                res3.Add(data);
            }
            stopwatch3.Stop();

            Console.WriteLine($"Time taken by Parallel.Foreach :{stopwatch1.Elapsed}  -  Count:{res1.Count}");
            Console.WriteLine($"Time taken by Task             :{stopwatch2.Elapsed}  -  Count:{res2.Count}");
            Console.WriteLine($"Time taken by Foreach          :{stopwatch3.Elapsed}  -  Count:{res3.Count}");

        }

如有任何帮助或建议，我将不胜感激，我想使用基于 Task 的并行调用，因为它在所有

方面都提供了最佳性能

Answer 1

您不能将 Parallel 与 async 一起使用； Parallel 只懂同步代码。

因此，最好的办法是使用异步并发，即调用异步方法并使用 Task.WhenAll 组合结果。如果工作是异步的，则 Task.Run 不是必需的。此外，return 结果比将它们作为副作用保存到数据结构中更自然。

Stopwatch stopwatch2 = Stopwatch.StartNew();

var listtask = countlist
    .Select(async () => await httpClient.GetAsync("https://jsonplaceholder.typicode.com/todos/1"))
    .ToList();
res2 = await Task.WhenAll(listtask);

stopwatch2.Stop();

Answer 2

使用 Task 方法发布最终答案对我有用，感谢@FuriousCactus 和@sellotape 提供了正确的路径。由于 list 不是线程安全的，这就是为什么很多它导致问题 ConcurrentBag 它解决了问题。

async static Task Main(string[] args)
        {
            HttpClient httpClient = new HttpClient();                    
            var res2 = new ConcurrentBag<HttpResponseMessage>();    
            var countlist = new List<int>();    
            for (int i = 1; i <= 50; i++)
                countlist.Add(i);

            // Task parallel
            Stopwatch stopwatch2 = Stopwatch.StartNew();

            var listtask = new List<Task>();
            foreach (var item in countlist)
            {
                listtask.Add(Task.Run(async () =>
                {
                    var data = await httpClient.GetAsync("https://jsonplaceholder.typicode.com/todos/1");
                    res2.Add(data);
                }));
            }
            await Task.WhenAll(listtask);
            stopwatch2.Stop();
         
            Console.WriteLine($"Time taken by Task             :{stopwatch2.Elapsed}  -  Count:{res2.Count}");

        }

Parallel.ForEach和任务请求处理请求失败问题

Parallel.ForEach and Task request processing request failure issue

c#

parallel-processing

task