SharePoint CSOM 请求挂起

Question

这种情况很少见，但已经出现过好几次了。这次下载文件的请求卡住了，我们的软件已经挂了将近 17 个小时。我们获取了一个 ProcDump，它显示停顿发生在“Socket.Receive”调用期间，这是 Microsoft.SharePoint.Client.ClientContext.ExecuteQuery 调用的结果。

这里是 ProcDump 堆栈跟踪：

ChildEBP RetAddr  Caller, Callee
08a5de44 6f45a7b5 mswsock!SockWaitForSingleObject+0x125, calling ntdll_77a80000!NtWaitForSingleObject
08a5dea0 6f46c7a8 mswsock!WSPRecv+0x2e8, calling mswsock!SockWaitForSingleObject
08a5df14 75681560 ws2_32!recv+0x100
08a5df68 72bccbff (MethodDesc 729f4eb4 +0x3b DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr, Byte*, Int32, System.Net.Sockets.SocketFlags))
08a5df90 72bccbff (MethodDesc 729f4eb4 +0x3b DomainBoundILStubClass.IL_STUB_PInvoke(IntPtr, Byte*, Int32, System.Net.Sockets.SocketFlags))
08a5dfac 72b6c3dd (MethodDesc 729e2b04 +0xbd System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError ByRef)), calling 72aed3fc
08a5dfd4 72b6c2ce (MethodDesc 729e2af8 +0x1e System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags)), calling (MethodDesc 729e2b04 +0 System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags, System.Net.Sockets.SocketError ByRef))
08a5dffc 72b6c203 (MethodDesc 729e0590 +0x83 System.Net.Sockets.NetworkStream.Read(Byte[], Int32, Int32)), calling (MethodDesc 729e2af8 +0 System.Net.Sockets.Socket.Receive(Byte[], Int32, Int32, System.Net.Sockets.SocketFlags))
08a5e030 72b5335d (MethodDesc 729e0ad0 +0x21 System.Net.FixedSizeReader.ReadPacket(Byte[], Int32, Int32))
08a5e04c 72b9b65e (MethodDesc 72a3fcb8 +0xa2 System.Net.Security._SslStream.StartFrameBody(Int32, Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)), calling (MethodDesc 729e0ad0 +0 System.Net.FixedSizeReader.ReadPacket(Byte[], Int32, Int32))
08a5e06c 72b9b3ba (MethodDesc 72a3fcac +0x92 System.Net.Security._SslStream.StartFrameHeader(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)), calling (MethodDesc 72a3fcb8 +0 System.Net.Security._SslStream.StartFrameBody(Int32, Byte[], Int32, Int32, System.Net.AsyncProtocolRequest))
08a5e094 72b9b087 (MethodDesc 72a3fca0 +0x77 System.Net.Security._SslStream.StartReading(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)), calling (MethodDesc 72a3fcac +0 System.Net.Security._SslStream.StartFrameHeader(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest))
08a5e0bc 72b9af60 (MethodDesc 729e0374 +0xc8 System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest)), calling (MethodDesc 72a3fca0 +0 System.Net.Security._SslStream.StartReading(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest))
08a5e0fc 72b532f3 (MethodDesc 72a4170c +0x4f System.Net.TlsStream.Read(Byte[], Int32, Int32)), calling (MethodDesc 729e0374 +0 System.Net.Security._SslStream.ProcessRead(Byte[], Int32, Int32, System.Net.AsyncProtocolRequest))
08a5e12c 72b6c0d0 (MethodDesc 72a3d738 +0x68 System.Net.PooledStream.Read(Byte[], Int32, Int32))
08a5e164 73216b6c (MethodDesc 72a3aa78 System.Net.ChunkParser.HandlePayload())
08a5e188 72b8ec41 (MethodDesc 729db7ec +0x3d System.Net.ChunkParser.ProcessResponse()), calling (MethodDesc 72a3aa78 +0 System.Net.ChunkParser.HandlePayload())
08a5e19c 730428a7 (MethodDesc 72a3aa24 +0x2f System.Net.ChunkParser.Read(Byte[], Int32, Int32)), calling (MethodDesc 729db7ec +0 System.Net.ChunkParser.ProcessResponse())
08a5e1c0 7320b446 (MethodDesc 72a3ee4c System.Net.ConnectStream.ReadWithoutValidation(Byte[], Int32, Int32, Boolean)), calling (MethodDesc 72a3aa24 +0 System.Net.ChunkParser.Read(Byte[], Int32, Int32))
08a5e1f4 72b6d885 (MethodDesc 729dff68 +0xe5 System.Net.ConnectStream.Read(Byte[], Int32, Int32)), calling (MethodDesc 72a3ee4c +0 System.Net.ConnectStream.ReadWithoutValidation(Byte[], Int32, Int32, Boolean))
08a5e230 013b8087 (MethodDesc 013849a0 +0xb7 Microsoft.SharePoint.Client.Mime.BufferedReadStream.Read(Byte[], Int32, Int32))
08a5e250 013b7eb2 (MethodDesc 0826e348 +0x2a Microsoft.SharePoint.Client.Mime.DelimitedStreamReader.Read(DelimitedReadStream, Byte[], Int32, Int32))
08a5e26c 013b7d08 (MethodDesc 0826e6a0 +0x48 Microsoft.SharePoint.Client.Mime.DelimitedStreamReader+DelimitedReadStream.Read(Byte[], Int32, Int32)), calling (MethodDesc 0826e348 +0 Microsoft.SharePoint.Client.Mime.DelimitedStreamReader.Read(DelimitedReadStream, Byte[], Int32, Int32))
08a5e28c 013b9c95 (MethodDesc 08d5ef3c +0x45 Microsoft.SharePoint.Client.ChunkStreamBuilder.CopyFrom(System.IO.Stream))
08a5e2a8 096da950 (MethodDesc 08d5abf8 +0x2a0 Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()), calling (MethodDesc 08d5ef3c +0 Microsoft.SharePoint.Client.ChunkStreamBuilder.CopyFrom(System.IO.Stream))
08a5e2f8 096da228 (MethodDesc 08d5ac1c +0xc8 Microsoft.SharePoint.Client.ClientRequest.ExecuteQueryToServer(Microsoft.SharePoint.Client.ChunkStringBuilder)), calling (MethodDesc 08d5abf8 +0 Microsoft.SharePoint.Client.ClientRequest.ProcessResponse())
08a5e324 08d4cf9d (MethodDesc 08d5956c +0x18d Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()), calling (MethodDesc 08d5ac1c +0 Microsoft.SharePoint.Client.ClientRequest.ExecuteQueryToServer(Microsoft.SharePoint.Client.ChunkStringBuilder))
...our code which calls Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()

有什么我们可以用它做的吗？或者我们应该依靠微软在他们的 CSOM 库中解决这个问题？

注意：我们使用 Microsoft.SharePointOnline.CSOM 库 (https://www.nuget.org/packages/Microsoft.SharePointOnline.CSOM/16.1.19927.12000) 的版本 16.1.19927.12000。

RequestTimeout 未明确设置，因此应使用默认值（3 分钟）。我们通常会时不时地看到超时，这是正常的。但是我们不希望看到一个永远挂起的请求。

编辑：

关于这个问题，我有一些更新。上次发生时，我们没有停止进程，而是决定等待。经过 32.5 小时 的等待，代码终于抛出 IOException，我们能够完成该过程。

下载文件请求已于 2021-08-25 发送 16:29:33.6426877.
代码在“System.Net.Sockets.Socket.Receive”行卡住了 32.5 小时。
2021-08-27 抛出异常01:11:23.5161189

IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

我们正在使用 .NET 4.6.1。

Answer 1

没有任何代码示例，我将不得不做出一些使用假设。如果您以与此类似的方式使用它，我们至少可以使用取消令牌来打破停滞。然后在捕获中你可以按照你认为合适的方式处理它。它可能就像重试一样简单，但如果不调试行为就很难说。

    try
    {
        var cancelToken = new CancellationTokenSource();
        var ms = 600_000;
        cancelToken.CancelAfter(ms);
        await ExecuteQueryWithTimeout(cancelToken);
    }
    catch (TaskCanceledException)
    {
        Console.WriteLine("Tasks cancelled: timed out");
    }
    finally
    {
        cancelToken.Dispose();
    }

ExecuteQuery 的异步方法

private static async Task ExecuteQueryWithTimeout(CancellationToken token)
{
    await Task.Run(()=> {
      Uri uri = new Uri(folderUrl);
      SP.Folder folder = web.GetFolderByServerRelativeUrl(uri.AbsolutePath);
      context.Load(folder);
      context.ExecuteQuery();
    });
}

Answer 2

我能想到几个原因：-

正在通过 TLS 1.0 建立连接。尝试安装 OS 更新，更新 .NET 框架，通过显式设置通过 TLS 1.2 建立连接。参考：https://docs.microsoft.com/en-us/dotnet/framework/network-programming/tls

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;

检查防火墙规则以确保它们没有阻止连接

Answer 3

最近，我通过 CSOM 方法在 SharePoint 中查询大量文件递归地抛出文件夹和子文件夹（超过 9000 个文件）时遇到了这个问题，经过长时间的研究，我遇到了这个 Microsoft 文档，其中包含解释以及如何解决这个问题：Avoid getting throttled or blocked in SharePoint Online

我使用这篇 Microsoft Docs 文章建议的扩展方法提取了下面的代码。

public static class CsomExtensions
{
    public static void ExecuteQueryWithIncrementalRetry(this ClientContext clientContext, int retryCount, int delay)
    {
        int retryAttempts = 0;
        int backoffInterval = delay;
        int retryAfterInterval = 0;
        bool retry = false;
        ClientRequestWrapper wrapper = null;
        if (retryCount <= 0)
            throw new ArgumentException("Provide a retry count greater than zero.");
        if (delay <= 0)
            throw new ArgumentException("Provide a delay greater than zero.");

        // Do while retry attempt is less than retry count
        while (retryAttempts < retryCount)
        {
            try
            {
                if (!retry)
                {
                    clientContext.ExecuteQuery();
                    return;
                }
                else
                {
                    //increment the retry count
                    retryAttempts++;

                    // retry the previous request using wrapper
                    if (wrapper != null && wrapper.Value != null)
                    {
                        clientContext.RetryQuery(wrapper.Value);
                        return;
                    }
                    // retry the previous request as normal
                    else
                    {
                        clientContext.ExecuteQuery();
                        return;
                    }
                }
            }
            catch (WebException ex)
            {
                var response = ex.Response as HttpWebResponse;
                // Check if request was throttled - http status code 429
                // Check is request failed due to server unavailable - http status code 503
                if (response != null && (response.StatusCode == (HttpStatusCode)429 || response.StatusCode == (HttpStatusCode)503))
                {
                    wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];
                    retry = true;

                    // Determine the retry after value - use the `Retry-After` header when available
                    string retryAfterHeader = response.GetResponseHeader("Retry-After");
                    if (!string.IsNullOrEmpty(retryAfterHeader))
                    {
                        if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))
                        {
                            retryAfterInterval = backoffInterval;
                        }
                    }
                    else
                    {
                        retryAfterInterval = backoffInterval;
                    }

                    // Delay for the requested seconds
                    Thread.Sleep(retryAfterInterval * 1000);

                    // Increase counters
                    backoffInterval = backoffInterval * 2;
                }
                else
                {
                    throw;
                }
            }
        }
        throw new MaximumRetryAttemptedException($"Maximum retry attempts {retryCount}, has be attempted.");
    }
}

[Serializable]
public class MaximumRetryAttemptedException : Exception
{
    public MaximumRetryAttemptedException(string message) : base(message) { }
}

SharePoint CSOM 请求挂起

SharePoint CSOM request hangs

.net

c#

sockets

sharepoint

csom