Parallel For 在 2.3 分钟后导致大量滞后峰值

Question

编辑：
我注意到这些滞后峰值仅在 visual studio 中调试时出现。如果我运行 Visual Stduio 之外的 .exe，程序不会使用超过 3% CPU.Can 谁能告诉我为什么会这样？

我在 Parallel processing.I 使用 Parallel.For 检查大量代理时遇到问题（通过发出 webrequests）。这是我的功能：

private ConcurrentBag<string> TotalProxies = new ConcurrentBag<string>();
private void CheckProxies()
{
    ParallelOptions pOptions = new ParallelOptions();
    pOptions.MaxDegreeOfParallelism = 100;
    int max = TotalProxies.Count;
    Invoke(new Action(() => { lbl_Status.Text = "Checking"; }));
    Parallel.For(0, max, pOptions, (index, loopstate) =>
    {
        string Proxy = TotalProxies.ElementAt(index);
        if (WebEngine.IsProxyWorking(Proxy))
        {
            WorkingProxies.Add(Proxy);
            workingp++;
            Invoke(new Action(() =>
            {
                lstv_Working.Items.Add(Proxy);
                lbl_Working.Text = workingp.ToString();
            }));
        }
        checkedp++;
        Invoke(new Action(() => { lbl_Checked.Text = checkedp.ToString(); }));

        if (Stop)
            loopstate.Stop();
    });
    Invoke(new Action(() => {
        lbl_Status.Text = "Idle";
    }));
}

我的问题如下：
该程序在前 0-2000 个请求中运行良好，其中 cpu 使用率约为 3-5%。然后，在 2-3 分钟后，我遇到了大量且频繁的滞后峰值，导致 CPU 使用率跃升至100%。我不知道为什么会发生这种情况，因为在 now.I 之前它一直运行良好，希望有人能帮助我了解导致这种情况的原因。
在这里你可以看到我的问题：

Answer 1

我不知道这是否与您的问题直接相关，但是将 MaxDegreeOfParallelism 设置为 100 并不好。您基本上是在告诉您的应用程序同时执行 100 个任务！根据MSDN

Generally, you do not need to modify this setting. However, you may choose to set it explicitly in advanced usage scenarios such as these:

When you know that a particular algorithm you're using won't scale beyond a certain number of cores. You can set the property to avoid wasting cycles on additional cores.

When you're running multiple algorithms concurrently and want to manually define how much of the system each algorithm can utilize. You can set a P:System.Threading.Tasks.ParallelOptions.MaxDegreeOfParallelism value for each.

When the thread pool's heuristics is unable to determine the right number of threads to use and could end up injecting too many threads. For example, in long-running loop body iterations, the thread pool might not be able to tell the difference between reasonable progress or livelock or deadlock, and might not be able to reclaim threads that were added to improve performance. In this case, you can set the property to ensure that you don't use more than a reasonable number of threads.

我会尝试删除该值并查看您的应用程序的行为方式！

Answer 2

正如 async/await 所承诺的示例，虽然看到您的更新我不确定它是否会有所作为。但由于它不适合评论，post在此处编辑 ;)

private ConcurrentBag<string> TotalProxies = new ConcurrentBag<string>();
private async Task CheckProxies()
{
    lbl_Status.Text = "Checking"; //NB, invoking is omitted assuming that CheckProxies is called from the UI thread itself
    var tasks = TotalProxies.Select(CheckProxy);
    await Task.WhenAll(tasks);
    lbl_Status.Text = "Idle";
}

private async Task<bool> CheckProxy(string p)
{   
    bool working = await Task.Run(() => WebEngine.IsProxyWorking(p)); //would be better if IsProxyWorking itself uses async methods and returns a task, so Task.Run isn't needed. Don't know if it's possible to alter that function?
    if(working)
    {
        WorkingProxies.Add(p);
        workingp++; //Interlocked.Increment is not necessary because after the await we're back in the main thread
        lstv_Working.Items.Add(p);  //are these items cleared on a new run? 
        lbl_Working.Text = workingp.ToString();
    }
    checkedp++;
    lbl_Checked.Text = checkedp.ToString(); 
    return working;
}

注意，由于我无法测试实际代码，所以我不确定效率。您当前的代码可能会执行得更好。但是，如果 IsProxyWorking 方法可以使用实际的异步网络调用（我相信该代码之前包含在您的 post 中），我相信处理会真正得到改善。

Parallel For 在 2.3 分钟后导致大量滞后峰值

Parallel For causes massive lag spikes after 2,3 minutes

c#

multithreading

webrequest

cpu-usage

parallel.for