express.js: 将上传的图片传递给 s3

Question

我正在尝试将从 React 应用上传的图像通过 express 传递到托管的 s3 存储桶。我正在使用的 platform/host 创建和管理 s3 存储桶并生成上传和访问 urls。这一切都很好（我已经在邮递员中测试了生成的上传 url 和二进制正文中的图像并且它工作得很好）。

我的问题是通过快递传递图像。我正在使用 multer 从表单中获取图像，但我假设 multer 正在将该图像转换为某种文件对象，而 s3 需要某种 blob 或流。

在下面的代码中，req.file 中的图像存在，我从 s3 收到 200 响应，没有错误，当我访问资产 url 时，url 有效，但是图片本身丢失。

const router = Router();
const upload = multer()

router.post('/', upload.single('file'), async (req, res) => {
    console.log(req.file)
    const asset = req.file
    const assetPath = req.headers['asset-path']

    let s3URLs = await getPresignedURLS(assetPath)
  
    const sendAsset = await fetch(
      s3URLs.urls[0].upload_url, // the s3 upload url 
      {
        method: 'PUT',
        headers: {
            "Content-Type": asset.mimetype
        },
        body: asset,
        redirect: 'follow'
      }
    )
  console.log("s3 response", sendAsset)

  res.status(200).json({"url": s3URLs.urls[0].access_url });
  
});

export default router;

我不确定如何将 multer 给我的内容转换为 aws s3 可以接受的内容。如果有更简单的方法来上传二进制文件以表达，我也愿意摆脱 multer。

Answer 1

您可以使用 multiparty to get file data from request object. And to upload file to s3 bucket you can use aws-sdk.

而不是 multer

const AWS = require("aws-sdk");
const multiparty = require("multiparty");

/**
 * Helper method which takes the request object and returns a promise with a data.
 */
const getDataFromRequest = (req) =>
  new Promise(async(resolve, reject) => {
    const form = new multiparty.Form();
    await form.parse(req, (err, fields, files) => {
      if (err) reject(err);
      const bucketname = fields.bucketname[0];
      const subfoldername = fields.subfoldername[0];
      const file = files["file"][0]; // get the file from the returned files object
      if (!file) reject("File was not found in form data.");
      else resolve({
        file,
        bucketname,
        subfoldername
      });
    });
  });

/**
 * Helper method which takes the request object and returns a promise with the AWS S3 object details.
 */
const uploadFileToS3Bucket = (
  file,
  bucketname,
  subfoldername,
  options = {}
) => {
  const s3 = new AWS.S3();

  // turn the file into a buffer for uploading
  const buffer = readFileSync(file.path);

  var originalname = file.originalFilename;
  var attach_split = originalname.split(".");
  var name = attach_split[0];
  // generate a new random file name
  const fileName = name;

  // the extension of your file
  const extension = extname(file.path);

  console.log(`${fileName}${extension}`);

  const params = {
    Bucket: bucketname, //Bucketname
    ACL: "private", //Permission
    Key: join(`${subfoldername}/`, `${fileName}${extension}`), // File name you want to save as in S3
    Body: buffer, // Content of file
  };

  // return a promise
  return new Promise((resolve, reject) => {
    return s3.upload(params, (err, result) => {
      if (err) reject(err);
      else resolve(result); // return the values of the successful AWS S3 request
    });
  });
};

router.post('/', upload.single('file'), async(req, res) => {
  try {
    // extract the file from the request object
    const {
      file,
      bucketname,
      subfoldername
    } = await getDataFromRequest(req);

    // Upload File to specified bucket
    const {
      Location,
      ETag,
      Bucket,
      Key
    } = await uploadFileToS3Bucket(
      file,
      bucketname,
      subfoldername
    );

    let response = {};
    res["Location"] = Location;
    response["ETag"] = ETag;
    response["Bucket"] = Bucket;
    response["Key"] = Key;

    res.status(200).json(response);
  } catch (error) {
    throw error;
  }

});

请求正文将是包含以下字段的表单数据

存储桶名称：
子文件夹名称：
文件：文件数据

Answer 2

对于任何偶然发现这个问题的人来说，解决方案是创建一个 custom multer storage engine。在引擎内部，您可以使用 s3 接受的流属性访问文件（使用正确的 headers）。

express.js: 将上传的图片传递给 s3

express.js: pass an uploaded image to s3

javascript

amazon-s3

node.js

express

multer