此代码在创建 zip 文件时可能损坏几个文件的任何原因
Any reason why this code may have corrupted couple of file while creating zip file
以下代码通过将它们拉入内存并将最终产品写入磁盘上的文件来从 S3 创建一个 zip 文件。但是,据观察者它在创建 zip 时损坏了几个文件(数千个)。我已经检查过,在此过程中损坏的文件没有任何问题,因为相同的文件可以通过其他方式正确压缩。对微调代码有什么建议吗?
代码:
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
FileStream fileStream = new FileStream(zipPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
using (ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Update, true))
{
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
zipEntryStream.Seek(0, SeekOrigin.Begin);
zipEntryStream.CopyTo(fileStream);
}
archive.Dispose();
fileStream.Close();
}
}
不要显式调用 Dispose()
或 Close()
,让 using
完成所有工作。而且你不需要写任何东西到 fileStream
写到 ZipArchiveEntry
stream 就可以在幕后完成它。您还需要使用 FileMode.Create
来保证您的文件在写入之前总是被截断。此外,由于您只创建存档而不更新它,您应该使用 ZipArchiveMode.Create
启用内存高效流式传输(感谢 @canton7 对 zip 存档格式的详细信息进行了深入研究)。
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
using FileStream fileStream = new FileStream(zipPath, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
using ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Create, true);
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
}
}
以下代码通过将它们拉入内存并将最终产品写入磁盘上的文件来从 S3 创建一个 zip 文件。但是,据观察者它在创建 zip 时损坏了几个文件(数千个)。我已经检查过,在此过程中损坏的文件没有任何问题,因为相同的文件可以通过其他方式正确压缩。对微调代码有什么建议吗?
代码:
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
FileStream fileStream = new FileStream(zipPath, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
using (ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Update, true))
{
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
zipEntryStream.Seek(0, SeekOrigin.Begin);
zipEntryStream.CopyTo(fileStream);
}
archive.Dispose();
fileStream.Close();
}
}
不要显式调用 Dispose()
或 Close()
,让 using
完成所有工作。而且你不需要写任何东西到 fileStream
写到 ZipArchiveEntry
stream 就可以在幕后完成它。您还需要使用 FileMode.Create
来保证您的文件在写入之前总是被截断。此外,由于您只创建存档而不更新它,您应该使用 ZipArchiveMode.Create
启用内存高效流式传输(感谢 @canton7 对 zip 存档格式的详细信息进行了深入研究)。
public static async Task S3ToZip(List<string> pdfBatch, string zipPath, IAmazonS3 s3Client)
{
using FileStream fileStream = new FileStream(zipPath, FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
using ZipArchive archive = new ZipArchive(fileStream, ZipArchiveMode.Create, true);
foreach (var file in pdfBatch)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = "sample-bucket",
Key = file
};
using GetObjectResponse response = await s3Client.GetObjectAsync(request);
using Stream responseStream = response.ResponseStream;
ZipArchiveEntry zipFileEntry = archive.CreateEntry(file.Split('/')[^1]);
using Stream zipEntryStream = zipFileEntry.Open();
await responseStream.CopyToAsync(zipEntryStream);
}
}