大文件上传到 Azure 文件存储失败

Question

尝试上传大于 4MB 的文件会引发 RequestBodyTooLarge 异常并显示以下消息：

The request body is too large and exceeds the maximum permissible limit.

虽然此限制记录在 REST API 参考 (https://docs.microsoft.com/en-us/rest/api/storageservices/put-range) it is not documented for the SDK Upload* methods (https://docs.microsoft.com/en-us/dotnet/api/azure.storage.files.shares.sharefileclient.uploadasync?view=azure-dotnet) 中。也没有解决此问题的示例。

那么如何上传大文件呢？

Answer 1

经过反复试验，我创建了以下方法来绕过文件上传限制。在下面的代码中 _dirClient 是一个已经初始化的 ShareDirectoryClient 设置为我要上传到的文件夹。

如果传入流大于 4MB，代码会从中读取 4MB 块并上传它们直到完成。 HttpRange 是将字节添加到已上传到 Azure 的文件的位置。必须增加索引以指向 Azure 文件的末尾，以便追加新字节。

public async Task WriteFileAsync(string filename, Stream stream) {

    //  Azure allows for 4MB max uploads  (4 x 1024 x 1024 = 4194304)
    const int uploadLimit = 4194304;

    stream.Seek(0, SeekOrigin.Begin);   // ensure stream is at the beginning
    var fileClient = await _dirClient.CreateFileAsync(filename, stream.Length);

    // If stream is below the limit upload directly
    if (stream.Length <= uploadLimit) {
        await fileClient.Value.UploadRangeAsync(new HttpRange(0, stream.Length), stream);
        return;
    }

    int bytesRead;
    long index = 0;
    byte[] buffer = new byte[uploadLimit];

    // Stream is larger than the limit so we need to upload in chunks
    while ((bytesRead = stream.Read(buffer, 0, buffer.Length)) > 0) {
        // Create a memory stream for the buffer to upload
        using MemoryStream ms = new MemoryStream(buffer, 0, bytesRead);
        await fileClient.Value.UploadRangeAsync(ShareFileRangeWriteType.Update, new HttpRange(index, ms.Length), ms);
        index += ms.Length; // increment the index to the account for bytes already written
    }
}

Answer 2

如果要将较大的文件上传到文件共享或 blob 存储，可以使用 Azure Storage Data Movement Library。

它提供high-performance上传、下载更大的文件。请考虑将此库用于更大的文件。

大文件上传到 Azure 文件存储失败

Uploading to Azure File Storage fails with large files

c#

azure

azure-files