IEnumerator 方法在后台计算下一个结果以加快响应速度

Question

我有一个方法returns IEnumerator，它对每条记录都有很长的计算过程。我怎样才能让它不完全卡在它的 yield return 命令上，而是在后台进行下一条记录计算以便更快地做出下一次响应？我不太关心线程安全，因为此方法的使用者在不同的 class 中，并且这两个 class 彼此非常隔离。

private int[] numbers = new int[] { 45, 43, 76, 23, 54, 22 };

private static int GetFibonacci(int n)
{
    if (n == 0 || n == 1) return n;
    else return GetFibonacci(n - 1) + GetFibonacci(n - 2);
}

public IEnumerator<int> GetFibonaccies()
{
    foreach (int n in numbers)
    {
        int f = GetFibonacci(n); // long job
        yield return f; // << please do not be lazy and do not stuck
                        // here till next request, but calculate next
                        // number in background to quickly respond
                        // your next request
    }
}

Answer 1

我使用下面的代码运行对此案例进行了测试。 FibonacciTestAsync 中的想法是运行并行计算。

答案分为 C# 7 和 C# 8 部分。

使用 C# 7 时的回答

序列 40、41、42、43、44、45 的测试持续时间：

FibonacciTestAsync：~31 秒
FibonacciTestNonAsync：~76 秒

请注意，这些结果比下面 asp.net 核心部分的结果要慢得多。

环境：Windows10、VS 2017、.NET Framework 4.6.1、NUnit、Resharper

private static readonly int[] Numbers = { 40, 41, 42, 43, 44, 45 };

private static int GetFibonacci(int n)
{
    if (n == 0 || n == 1) return n;
    return GetFibonacci(n - 1) + GetFibonacci(n - 2);
}

private static Task<int> GetFibonacciAsync(int n) => Task.Run(() => GetFibonacci(n));

public static IEnumerator<int> GetFibonaccies()
{
    foreach (var n in Numbers)
    {
        var f = GetFibonacci(n); // long job
        yield return f; // << please do not be lazy and do not stuck here till next request but calculate next number in background to quickly respond your next request
    }
}

// in C# 8: public static async IAsyncEnumerable<int> GetFibonacciesAsync()
public static IEnumerable<int> GetFibonacciesAsync()
{
    var taskList = Numbers
        .Select(GetFibonacciAsync)
        .ToList();

    foreach (var task in taskList)
    {
        // in C# 8: yield return await task;
        yield return task.GetAwaiter().GetResult();
    }
}

private static readonly IList<int> ExpectedOutput = new List<int>
{
    102334155,
    165580141,
    267914296,
    433494437,
    701408733,
    1134903170
};

[Test]
public void FibonacciTestNonAsync()
{
    var sw = new Stopwatch();
    sw.Start();

    var result = new List<int>();

    using (var fibonacciNumberEnumerator = GetFibonaccies())
    {
        while (fibonacciNumberEnumerator.MoveNext())
        {
            result.Add(fibonacciNumberEnumerator.Current);
            Console.WriteLine(fibonacciNumberEnumerator.Current);
        }
    }

    sw.Stop();
    Console.WriteLine("Elapsed={0}", (double)sw.ElapsedMilliseconds / 1000);

    Assert.AreEqual(ExpectedOutput, result);
}

[Test]
public void FibonacciTestAsync()
{
    var sw = new Stopwatch();
    sw.Start();

    var result = new List<int>();

    // here you can play a little bit:
    // Try to replace GetFibonacciesAsync() with GetFibonacciesAsync().Take(1) and observe that the test will run a lot faster
    var fibonacciNumbers = GetFibonacciesAsync();
    foreach (var item in fibonacciNumbers)
    {
        result.Add(item);
        Console.WriteLine(item);
    }
    sw.Stop();
    Console.WriteLine("Elapsed={0}", (double)sw.ElapsedMilliseconds / 1000);

    Assert.AreEqual(ExpectedOutput, result);
}

使用 C# 8 或更高版本时的回答

序列 40、41、42、43、44、45 的测试持续时间：

FibonacciTestAsync：~11 秒
FibonacciTestNonAsync：~26 秒

测试期间使用的环境：Windows10、VS 2019、asp.netCore 5、NUnit、Resharper

// original array from OP, takes too long too compute private static readonly int[] Numbers = { 45, 43, 76, 23, 54, 22 };

private static readonly int[] Numbers = { 40, 41, 42, 43, 44, 45 };

private static int GetFibonacci(int n)
{
    if (n == 0 || n == 1) return n;
    return GetFibonacci(n - 1) + GetFibonacci(n - 2);
}

private static Task<int> GetFibonacciAsync(int n) => Task.Run(() => GetFibonacci(n));

public static IEnumerator<int> GetFibonaccies()
{
    foreach (int n in Numbers)
    {
        var f = GetFibonacci(n); // long job
        yield return f; // << please do not be lazy and do not stuck here till next request but calculate next number in background to quickly respond your next request
    }
}

public static async IAsyncEnumerable<int> GetFibonacciesAsync()
{
    var taskList = Numbers
        .Select(GetFibonacciAsync) // starting task here
        .ToList();

    foreach (var task in taskList)
    {
        yield return await task; // as soon as current task is completed, yield the result
    }
}

private static readonly IList<int> ExpectedOutput = new List<int>
{
    102334155,
    165580141,
    267914296,
    433494437,
    701408733,
    1134903170
};

[Test]
public void FibonacciTestNonAsync()
{
    var sw = new Stopwatch();
    sw.Start();

    var result = new List<int>();

    using IEnumerator<int> fibonacciNumberEnumerator = GetFibonaccies();
    while (fibonacciNumberEnumerator.MoveNext())
    {
        result.Add(fibonacciNumberEnumerator.Current);
        Console.WriteLine(fibonacciNumberEnumerator.Current);
    }

    sw.Stop();
    Console.WriteLine("Elapsed={0}", (double)sw.ElapsedMilliseconds / 1000);

    Assert.AreEqual(ExpectedOutput, result);
}

[Test]
public async Task FibonacciTestAsync()
{
    var sw = new Stopwatch();
    sw.Start();

    var result = new List<int>();

    var fibonacciNumbers = GetFibonacciesAsync();
    await foreach (var item in fibonacciNumbers)
    {
        result.Add(item);
        Console.WriteLine(item);
    }
    sw.Stop();
    Console.WriteLine("Elapsed={0}", (double)sw.ElapsedMilliseconds / 1000);

    Assert.AreEqual(ExpectedOutput, result);
}

Answer 2

这是 IEnumerable<T>s 的扩展方法 WithPreloadNext，它将下一个 MoveNext 调用卸载到 ThreadPool，同时将先前的值交给调用者：

public static IEnumerable<T> WithPreloadNext<T>(this IEnumerable<T> source)
{
    // Argument validation omitted
    using var enumerator = source.GetEnumerator();
    Task<(bool, T)> task = Task.Run(() => enumerator.MoveNext() ?
        (true, enumerator.Current) : (false, default));
    while (true)
    {
        var (moved, value) = task.GetAwaiter().GetResult();
        if (!moved) break;
        task = Task.Run(() => enumerator.MoveNext() ?
            (true, enumerator.Current) : (false, default));
        yield return value;
    }
}

用法示例：

private IEnumerable<int> GetFibonacciesInternal()
{
    foreach (int n in numbers) yield return GetFibonacci(n);
}

public IEnumerable<int> GetFibonaccies() => GetFibonacciesInternal().WithPreloadNext();

注意： 卸载 MoveNext 意味着源可枚举不会在调用者的上下文中枚举。因此，如果源可枚举对象与当前线程具有线程关联性，则不应使用此方法。例如，在 Windows Forms 应用程序的情况下，可枚举与 UI 组件交互。

IEnumerator 方法在后台计算下一个结果以加快响应速度

IEnumerator method to calculate next result in background for quicker response

c#

parallel-processing

ienumerable

ienumerator

yield-return