带有 etags 的 s3 流式传输（节点）

Question

我们在 aws-sdk 中使用 createReadStream() 在节点中流式传输来自 S3 的内容。我们想添加 etag 支持。如果我们从客户端添加 'If-None-Match' header，s3 将 NotModified 作为我似乎无法处理的错误抛出。

retrieveFile = function(req, res) {
    var s3 = new AWS.S3(); 
    var params = {
        Bucket: bucket,
        key: key
    };
    if (req.get('If-None-Match')) {
        params.IfNoneMatch = req.get('If-None-Match');
    }
    return s3.getObject(params).on('httpHeaders', function(statusCode, headers) {
        if (headers.etag) {
            res.set('etag', headers.etag);
        }
        if (headers['content-length']) {
            return res.set('content-length', headers['content-length']);
        }
    }).createReadStream().pipe(res);
};

我试过监听流上的事件并对从 getObject 返回的请求使用回调。我可以通过这种方式获得错误消息，但 aws-sdk 中的其他内容似乎正在终止我的进程。

/Projects/my-app/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

NotModified: null
  at Request.extractError (/Projects/my-app/node_modules/aws-sdk/lib/services/s3.js:519:35)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
  at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
  at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
  at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
  at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
  at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
  at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
  at callNextListener (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
  at IncomingMessage.onEnd (/Projects/my-app/node_modules/aws-sdk/lib/event_listeners.js:244:11)
  at emitNone (events.js:91:20)
  at IncomingMessage.emit (events.js:185:7)
  at endReadableNT (_stream_readable.js:974:12)
  at _combinedTickCallback (internal/process/next_tick.js:74:11)
  at process._tickDomainCallback (internal/process/next_tick.js:122:9)

Answer 1

似乎错误是在流上抛出的，如果您正在监听 AWS.Request，您将收到一个错误，但仍然没有捕获到流错误。

考虑一下：

s3.getObject({
    Bucket: 'foo',
    Key: 'bar'
}).on('error', function (err) {
    console.log('Error event!');
}).createReadStream();

这将显示 'Error event!' 但它也会抛出错误并退出进程。那是因为 createReadStream 创建的流也收到错误。

考虑一下：

s3.getObject({
    Bucket: 'lalaland',
    Key: 'blabaliets'
}).on('error', function (err) {
    console.log('Error event!');
}).createReadStream().on('error', function (err) {
    console.log('Error event on stream!');
});

它将在 AWS.Request 对象上侦听错误，但 也会在流对象上侦听 。这将首先打印 'Error event!'，然后打印 'Error event on stream!'，进程不会退出。

注意这个：

s3.getObject({
    Bucket: 'lalaland',
    Key: 'blabaliets'
}).createReadStream().on('error', function (err) {
    console.log('Error event on stream!');
});

将仅打印'Error event on stream!'并且不会退出。最后一个选项可能是您想要的：只监听流上的错误，而不是请求对象上的错误。

回到初心；此时（在流上的 on error 中）你想检查它是否确实是 NotModified 错误并执行类似 res.status(304).end() 的操作。

带有 etags 的 s3 流式传输（节点）

s3 streaming with etags (node)

amazon-s3

amazon-web-services

node.js

aws-sdk

aws-sdk-nodejs