如何将大文件上传到 Azure Blob Storage (.NET Core)

Question

我尝试将 2GB 的文件上传到 Azure 存储，但我的代码失败并且出现异常。对于较小的文件，它工作正常。如何更改超时和文件大小限制？我在 .NET Core 上使用 Azure.Storage.Blobs 版本 12.10.0。

我的代码：

    public FileRepository(AppSettings appSettings)
    {
        var blobServiceClient = new BlobServiceClient(appSettings.StorageConnectionString);

        var containers = blobServiceClient.GetBlobContainers();
        if (!containers.Any(x => x.Name == containerName))
        {
            blobServiceClient.CreateBlobContainer(containerName);
        }
        _blobContainerClient = blobServiceClient.GetBlobContainerClient(containerName);
    }

    public async Task<string> Upload(string fileName, Stream content)
    {
        var blobName = $"{Guid.NewGuid()}/{fileName}";
        await _blobContainerClient.UploadBlobAsync(blobName, content);
        return blobName;
    }

例外我有：

System.AggregateException
  HResult=0x80131500
  Message=Retry failed after 6 tries. (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.) (The operation was canceled.)
  Source=Azure.Core
  StackTrace:
   at Azure.Core.Pipeline.RetryPolicy.<ProcessAsync>d__11.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.<ProcessAsync>d__1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Storage.Blobs.BlobRestClient.BlockBlob.<UploadAsync>d__0.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Storage.Blobs.Specialized.BlockBlobClient.<UploadInternal>d__26.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.Specialized.BlockBlobClient.<>c__DisplayClass48_0.<<GetPartitionedUploaderBehaviors>b__0>d.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.PartitionedUploader`2.<UploadInternal>d__19.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.BlobClient.<StagedUploadInternal>d__29.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Azure.Storage.Blobs.BlobContainerClient.<UploadBlobAsync>d__83.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at C:\...\FileRepository.<Upload>d__3.MoveNext() in C:\...\FileRepository.cs:line 38

Inner Exception 1:
TaskCanceledException: The operation was canceled.

Inner Exception 2:
HttpRequestException: Error while copying content to a stream.

Inner Exception 3:
IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..

Inner Exception 4:
SocketException: The I/O operation has been aborted because of either a thread exit or an application request.

Answer 1

您可以在客户端将文件拆分为 chunks/blocks。然后发送很多小块 100 MB。

将文件拆分成更小的块，如果您正在使用 Stream，代码将只读取一定数量的字节并处理这些字节直到完成。

请参考Implement File Chunking ,Upload a file in blocks 和 Put Blob 了解更多详情。

Answer 2

我认为使用BlobContainerClient上传时无法控制超时选项和文件大小限制。

您将需要使用 UploadAsync method in BlobClient class and specify appropriate values in BlobUploadOptions.TransferOptions。

如果您想要对上传过程进行更精细的控制，您将需要在特定的 blob 客户端中使用上传方法。例如，如果您要上传块 blob，则可以使用 BlockBlobClient.StageBlockAsync and BlockBlobClient.CommitBlockListAsync.

如何将大文件上传到 Azure Blob Storage (.NET Core)

How to upload big files to Azure Blob Storage (.NET Core)

azure

.net-core

azure-blob-storage

azure-storage