使用高级多部分上传仍然没有指定内容长度警告
Upload with high level multipart still gives no content length specified warning
即使我使用的是高级多部分,我仍然在控制台中收到警告:
WARN - com.amazonaws.services.s3.AmazonS3Client - No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.
这就是我使用高级分段上传的方式,就像这里一样:https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-upload-object.html
val tm: TransferManager = TransferManagerBuilder
.standard()
.withS3Client(s3Client)
.withMultipartUploadThreshold(5248000)
.build();
val metadata = new ObjectMetadata()
metadata.setContentType(mimeType)
val request = new PutObjectRequest(bucketName, key, inputStream, metadata)
val upload = tm.upload(request)
upload.waitForCompletion()
5248000 是 5MB,我尝试上传比这大得多的文件,所以它应该使用 withMultipartUploadThreshold
文档中所说的多部分策略:
Sets the size threshold, in bytes, for when to use multipart uploads.
Uploads over this size will automatically use a multipart upload
strategy, while uploads smaller than this threshold will use a single
connection to upload the whole object.
为什么还是出现这个警告?
documentation for the AWS SDK for Java 在 ObjectMetadata 上提到了这个:
This field is required when uploading objects to S3, but the Amazon Web Services S3 Java client will automatically set it when working directly with files. When uploading directly from a stream, set this field if possible. Otherwise the client must buffer the entire stream in order to calculate the content length before sending the data to Amazon S3.
换句话说,在构造 PutObjectRequest
对象之前,您需要使用文件或流大小显式调用 metadata.setContentLength(x)
。如果不这样做,AWS SDK 将需要在 ram 中缓冲整个流,可能会耗尽较大对象的内存,这会触发您看到的警告。
即使我使用的是高级多部分,我仍然在控制台中收到警告:
WARN - com.amazonaws.services.s3.AmazonS3Client - No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.
这就是我使用高级分段上传的方式,就像这里一样:https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-upload-object.html
val tm: TransferManager = TransferManagerBuilder
.standard()
.withS3Client(s3Client)
.withMultipartUploadThreshold(5248000)
.build();
val metadata = new ObjectMetadata()
metadata.setContentType(mimeType)
val request = new PutObjectRequest(bucketName, key, inputStream, metadata)
val upload = tm.upload(request)
upload.waitForCompletion()
5248000 是 5MB,我尝试上传比这大得多的文件,所以它应该使用 withMultipartUploadThreshold
文档中所说的多部分策略:
Sets the size threshold, in bytes, for when to use multipart uploads. Uploads over this size will automatically use a multipart upload strategy, while uploads smaller than this threshold will use a single connection to upload the whole object.
为什么还是出现这个警告?
documentation for the AWS SDK for Java 在 ObjectMetadata 上提到了这个:
This field is required when uploading objects to S3, but the Amazon Web Services S3 Java client will automatically set it when working directly with files. When uploading directly from a stream, set this field if possible. Otherwise the client must buffer the entire stream in order to calculate the content length before sending the data to Amazon S3.
换句话说,在构造 PutObjectRequest
对象之前,您需要使用文件或流大小显式调用 metadata.setContentLength(x)
。如果不这样做,AWS SDK 将需要在 ram 中缓冲整个流,可能会耗尽较大对象的内存,这会触发您看到的警告。