将 aws-sdk 升级到版本 3 - 流式 S3 上传

Question

我正在尝试将程序升级到 aws-sdk 版本 3，但出现错误：

NotImplemented: A header you provided implies functionality that is not implemented

函数 get_s3_stream 需要 return 到主程序的流。

SDK 版本 2，有效：

var AWS = require("aws-sdk");
var s3 = new AWS.S3({apiVersion: '2006-03-01'});
const stream = require('stream');

function get_s3_stream() {
  var pass = new stream.PassThrough();
  var params = {Bucket: "bucketname", Key: "testfile1.txt", Body: pass};
  s3.upload(params, function(err, data) {
    console.log(err, data);
  });
  return pass;
}

const inputStream = stream.Readable.from(["input string1"])
const outStream = get_s3_stream()
inputStream.pipe(outStream);

SDK 版本 3，不工作：

const { S3Client, PutObjectCommand } = require("@aws-sdk/client-s3");
const s3Client = new S3Client({ region: "us-east-1" });
const stream = require('stream')

function get_s3_stream() {
    const pass = new stream.PassThrough();  
    var params = {Bucket: "bucketname", Key: "testfile2.txt", Body: pass};
    s3Client.send(new PutObjectCommand(params, function(err, data) {
        console.log(err, data);
      }));
    return pass;
};

const inputStream = stream.Readable.from(["input string2"])
const outStream = get_s3_stream()
inputStream.pipe(outStream);

如何使用新版本将流发送到 S3？

Answer 1

显然这是一个已知问题：https://github.com/aws/aws-sdk-js-v3/issues/1920。

使用直通流的解决方法是使用 Upload from @aws-sdk/lib-storage:

const { S3Client } = require("@aws-sdk/client-s3");
const { Upload } = require('@aws-sdk/lib-storage');
const stream = require('stream')

const s3Client = new S3Client({ region: "us-east-1" });

function get_s3_stream() {
    const pass = new stream.PassThrough();
    const upload = new Upload({
        client: s3Client,
        params: {
            Bucket: 'bucketname',
            Key: 'testfile2.txt',
            Body: pass,
            ContentType: 'text/plain',
        },
    });

    upload.done().then((res, error) => {
        console.log(res);
    });

    return pass;
}

将 aws-sdk 升级到版本 3 - 流式 S3 上传

Upgrade aws-sdk to version 3 - streaming S3 upload

node.js

amazon-web-services

amazon-s3

streaming