for循环中的速率限制请求承诺

Question

我正在尝试使用 promises 来限制我向外部 API 发送 GET 请求的速率，但我很难让它正常工作。在我的场景中，我正在使用 'request-promise' 模块，我需要从 API 发送针对 175 个项目的 GET 请求（每个项目 ID 一个请求）。 API 的速率限制为每 10 秒 40 个请求，因此我的限制需要为每个请求 250 毫秒。我正在尝试为每个项目 ID 发送内部循环请求，例如：

files.forEach(function (file, i) {
    console.log("The item ID is " + file.match(re)[1]);
    client.send(new APIClient.requests.getItem(file.match(re)[1]))
    .then((item) => {
      ...
    })
    .catch((error) => {
      console.error(error);
      // Use fallback
    });
    ...

这是我的 API 客户端 return 请求承诺 (rp) 的片段，超时为 250 毫秒且无回调：

const rp = require('request-promise');
const rp_errors = require('request-promise/errors');
...
send(request, callback) {
...
    return rp(options)
               .then(this._parseResponse)
               .then((response)=> {
                  return new Promise( (resolve) => setTimeout(function(){
                    if (callback) { return callback(null, response); }
                    return resolve(response);
                  }, 250));
                })
                .catch(rp_errors.StatusCodeError,((error) => {
                    throw new errors.ResponseError(request, error.statusCode, error.message);
                  }
                ))
                .catch(rp_errors.RequestError,((error) => {
                    if(error.cause.code === 'ETIMEDOUT' || error.cause.code === 'ESOCKETTIMEDOUT')
                      throw new errors.TimeoutError(request, error);
                    throw error;
                  }
                ))
                .catch((error) => {
                  if (callback) {return callback(error)};
                  throw error;
                });
}

异步不起作用，return这是“超过 429 个请求限制”的堆栈跟踪

{ ResponseError: 429 - {"status_code":25,"status_message":"Your request count (175) is over the allowed limit of 40."}
[0]     at rp.then.then.catch (/mnt/c/Users/ridhwaan/Source/homehost/lib/api-client.js:52:19)
[0]     at tryCatcher (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/util.js:16:23)
[0]     at /mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/catch_filter.js:17:41
[0]     at tryCatcher (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/util.js:16:23)
[0]     at Promise._settlePromiseFromHandler (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/promise.js:512:31)
[0]     at Promise._settlePromise (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/promise.js:569:18)
[0]     at Promise._settlePromise0 (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/promise.js:614:10)
[0]     at Promise._settlePromises (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/promise.js:689:18)
[0]     at Async._drainQueue (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/async.js:133:16)
[0]     at Async._drainQueues (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/async.js:143:10)
[0]     at Immediate.Async.drainQueues [as _onImmediate] (/mnt/c/Users/ridhwaan/Source/homehost/node_modules/bluebird/js/release/async.js:17:14)
[0]     at runCallback (timers.js:756:18)
[0]     at tryOnImmediate (timers.js:717:5)
[0]     at processImmediate [as _immediateCallback] (timers.js:697:5)
[0]   name: 'ResponseError',
[0]   request:
[0]    Movie {
[0]      method: 'GET',
[0]      path: '/movie/24428',
[0]      timeout: 10000,
[0]      ensureHttps: false,
[0]      external_id: '24428' },
[0]   statusCode: 429 }

Answer 1

所以主要问题是 array.forEach 是一个同步函数，不会等待 client.send 完成。一种解决方案是使用 bluebird.mapSeries (http://bluebirdjs.com/docs/api/promise.mapseries.html) 映射数组并等待每次迭代完成。也不要忘记 return send-promise，所以 mapSeries-function 会知道它何时解决，因此它知道何时开始下一次迭代：

bluebird.mapSeries(files, function(file){
  return send(...)
}

最后一个建议是将整个 .then(... setTimeout...) 部分替换为 .delay(250)。 Request-Promise 已经使用了 bluebird promises，因此您可以使用它们的便捷功能 http://bluebirdjs.com/docs/api/delay.html。延迟会自动解析之前promise的值

return rp(options)
   .then(this._parseResponse)
   .delay(250)
   .catch(...)

for循环中的速率限制请求承诺

Rate limiting request-promise in for loop

node.js

promise

request-promise