解压缩 SQL Blob 内容和 return PushStreamContent .NET Core 中的答案

Question

我正在 .NET Core 服务中开发一个新的 API，新的 API 应该从 SQL table 中读取一个 BLOB，使用DeflateStream。然后 return 将其（流式传输）到客户端。

为了不占用太多内存。我正在 returning 类型的响应和 PushStreamContent 以便我可以将 sql 流直接复制到响应流中而无需将 blob 加载到内存中。所以我最终得到了类似的结果。

return this.ResponseMessage(new HttpResponseMessage
        {
            Content = new PushStreamContent(async (outStream, httpContent, transportContext) =>
            {
                using (SqlConnection connection = new SqlConnection(connectionString))
                {
                    await connection.OpenAsync();
                    using (SqlCommand command = new SqlCommand(query, connection))
                    {

                        // The reader needs to be executed with the SequentialAccess behavior to enable network streaming
                        // Otherwise ReadAsync will buffer the entire BLOB into memory which can cause scalability issues or even OutOfMemoryExceptions
                        using (SqlDataReader reader = await command.ExecuteReaderAsync(CommandBehavior.SequentialAccess))
                        {
                            if (await reader.ReadAsync() && !(await reader.IsDBNullAsync(0)))
                            {
                                using (Stream streamToDecompress = reader.GetStream(0))
                                using (Stream decompressionStream = new DeflateStream(streamToDecompress, CompressionMode.Decompress))
                                {
                                    // This copyToAsync will take for ever
                                    await decompressionStream.CopyToAsync(outStream);
                                    outStream.close();

                                    return;
                                }
                            }

                            throw new Exception("Couldn't retrieve blob");
                        }
                    }
                }
            },
            "application/octet-stream")
        });

这里的问题是，如代码中所述，将 deflateStream 复制到响应输出流的步骤需要永远。尽管我尝试了完全相同的方法，但将流写入文件而不是将其复制到 resp 流，而且效果非常好。

所以你们能帮我解决这个问题吗？？我使用 PushStreamContent 有错吗？我应该使用不同的方法吗？问题是我不想将整个 Blob 加载到内存中，我想读取它并即时解压。 SqlClient Supports streaming blob，我想利用它。

Answer 1

这是 PushStreamContent 中的一个死锁，我并不假装理解。但我重新制作并更改了

await decompressionStream.CopyToAsync(outStream);

至

decompressionStream.CopyTo(outStream);

解决它。

这是完整的重现：

public ResponseMessageResult Get()
{
    var data =  new string[] { "value1", "value2" };

    var jsonData = Newtonsoft.Json.JsonConvert.SerializeObject(data);

    var msSource = new MemoryStream(Encoding.UTF8.GetBytes(jsonData));
    var msDest = new MemoryStream();
    var compressionStream = new DeflateStream(msDest, CompressionMode.Compress);
    msSource.CopyTo(compressionStream);
    compressionStream.Close();

    var compressedBytes = msDest.ToArray();

    var query = "select @bytes buf";
    var connectionString = "server=localhost;database=tempdb;integrated security=true";

    
    return this.ResponseMessage(new HttpResponseMessage
    {
        Content = new PushStreamContent(async (outStream, httpContent, transportContext) =>
        {
            using (SqlConnection connection = new SqlConnection(connectionString))
            {
                await connection.OpenAsync();
                using (SqlCommand command = new SqlCommand(query, connection))
                {
                    command.Parameters.Add("@bytes", SqlDbType.VarBinary, -1).Value = compressedBytes;

                    // The reader needs to be executed with the SequentialAccess behavior to enable network streaming
                    // Otherwise ReadAsync will buffer the entire BLOB into memory which can cause scalability issues or even OutOfMemoryExceptions
                    using (SqlDataReader reader = await command.ExecuteReaderAsync(CommandBehavior.SequentialAccess))
                    {
                        if (await reader.ReadAsync() && !(await reader.IsDBNullAsync(0)))
                        {
                            using (Stream streamToDecompress = reader.GetStream(0))
                            {
                                //var buf = new MemoryStream();
                                //streamToDecompress.CopyTo(buf);
                                //buf.Position = 0;

                                using (Stream decompressionStream = new DeflateStream(streamToDecompress, CompressionMode.Decompress))
                                {
                                    
                                    // This copyToAsync will take for ever
                                    //await decompressionStream.CopyToAsync(outStream);
                                    decompressionStream.CopyTo(outStream);
                                    outStream.Close();

                                    return;
                                }
                            }
                        }

                        throw new Exception("Couldn't retrieve blob");
                    }
                }
            }
        },
    "application/octet-stream")
    });
}

解压缩 SQL Blob 内容和 return PushStreamContent .NET Core 中的答案

Decompress SQL Blob Content and return the answer in PushStreamContent .NET Core

.net

c#

sqlclient

deflatestream

.net-core