imgurpython.helpers.error.ImgurClientRateLimitError: Rate-limit exceeded
imgurpython.helpers.error.ImgurClientRateLimitError: Rate-limit exceeded
我有以下错误：
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Traceback (most recent call last):
File "download.py", line 22, in <module>
search = imgur_client.gallery_search('cat', window='all', sort='time', page=p)
File "/usr/local/lib/python2.7/dist-packages/imgurpython/client.py", line 531, in gallery_search
response = self.make_request('GET', 'gallery/search/%s/%s/%s' % (sort, window, page), data)
File "/usr/local/lib/python2.7/dist-packages/imgurpython/client.py", line 153, in make_request
raise ImgurClientRateLimitError()
imgurpython.helpers.error.ImgurClientRateLimitError: Rate-limit exceeded!
对于此代码：
1 from imgurpython import ImgurClient
2 import inspect
3 import random
4 import urllib2
5 import requests
6 from imgurpython.helpers.error import ImgurClientError
7
8 client_id = "ABC"
9 client_secret = "ABC"
10 access_token = "ABC"
11 refresh_token = "ABC"
12
13
14
15 image_type = ['jpg', 'jpeg']
16
17 imgur_client = ImgurClient(client_id, client_secret, access_token, refresh_token)
18
19 item_count = 0
20 for p in range(1, 10000):
21 try:
22 search = imgur_client.gallery_search('cat', window='all', sort='time', page=p)
23 for i in range(0,https://gist.github.com/monajalal/e02792e9a5cbced301a8691b7a62836f len(search)):
24 item_count +=1
25 print(search[i].comment_count)
26 if search[i].comment_count > 10 and not search[i].is_album:
27 print(search[i].type)
28 if search[i].type[6:] in image_type:
29 count = 0
30 try:
31 image_file = urllib2.urlopen(search[i].link, timeout = 5)
32 image_file_name = 'images/'+ search[i].id+'.'+search[i].type[6:]
33 output_image = open(image_file_name, 'wb')
34 output_image.write(image_file.read())
35 for post in imgur_client.gallery_item_comments(search[i].id, sort='best'):
36 if count <= 10:
37 count += 1
38 output_image.close()
39 except urllib2.URLError as e:
40 print(e)
41 continue
42 except socket.timeout as e:
43 print(e)
44 continue
45 except socket.error as e:
46 print(e)
47 continue
48 except ImgurClientError as e:
49 print(e)
50 continue
51
52 print item_count
我也经常看到这条线：
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
如何修复错误？ Imgur 中的速率限制错误是否有任何解决方法？所以我创建这个应用程序是为了学术研究而不是商业用途，根据 https://api.imgur.com/#limits 它应该是免费的，但我必须注册我的应用程序才能获得 client_id 相关的东西。如何将我的应用程序设置为非商业应用程序，这样我就不会收到此速率限制错误，或者如果所有类型的应用程序都收到此错误，我应该如何处理？我应该如何设置我的代码以使其每小时仅发出 1250 个请求？
还有我的信用信息：
User Limit: 500
User Remaining: 500
User Reset: 2016-10-18 14:32:41
User Client Limit: 12500
User Client Remaining: 9570
更新：按照答案中的建议使用 sleep(8)，我最终会持续进行下去。对于不同的搜索查询，这发生在不同的页面。我该如何修复代码，以便它在发生这种情况时停止执行？这是更新的相关代码：https://gist.github.com/monajalal/e02792e9a5cbced301a8691b7a62836f
page number is: 157
0
image/jpeg
page number is: 157
0
page number is: 157
0
page number is: 157
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
速率限制是指您点击 API 的频率，而不是您被允许的呼叫次数。为了防止锤击，大多数 API 都有速率限制（例如：每分钟 30 个请求，每 2 秒 1 个）。您的脚本正在尽可能快地发出请求，比限制快数百甚至数千倍。
为了防止您的脚本受到影响，最简单的解决方案是在您的 for 循环中引入一个 sleep。
from time import sleep
for i in range(10000):
print i
sleep(2) # seconds
将休眠时间调整为至少比 API 定义的速率限制长一秒。
The Imgur API uses a credit allocation system to ensure fair distribution of capacity. Each application can allow approximately 1,250 uploads per day or approximately 12,500 requests per day. If the daily limit is hit five times in a month, then the app will be blocked for the rest of the month. The remaining credit limit will be shown with each requests response in the X-RateLimit-ClientRemaining HTTP header.
所以 12500 个请求/24 小时是每小时 520 个请求或每分钟约 8 个请求。这意味着您的睡眠时间应约为 8 秒。
我有以下错误：
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Traceback (most recent call last):
File "download.py", line 22, in <module>
search = imgur_client.gallery_search('cat', window='all', sort='time', page=p)
File "/usr/local/lib/python2.7/dist-packages/imgurpython/client.py", line 531, in gallery_search
response = self.make_request('GET', 'gallery/search/%s/%s/%s' % (sort, window, page), data)
File "/usr/local/lib/python2.7/dist-packages/imgurpython/client.py", line 153, in make_request
raise ImgurClientRateLimitError()
imgurpython.helpers.error.ImgurClientRateLimitError: Rate-limit exceeded!
对于此代码：
1 from imgurpython import ImgurClient
2 import inspect
3 import random
4 import urllib2
5 import requests
6 from imgurpython.helpers.error import ImgurClientError
7
8 client_id = "ABC"
9 client_secret = "ABC"
10 access_token = "ABC"
11 refresh_token = "ABC"
12
13
14
15 image_type = ['jpg', 'jpeg']
16
17 imgur_client = ImgurClient(client_id, client_secret, access_token, refresh_token)
18
19 item_count = 0
20 for p in range(1, 10000):
21 try:
22 search = imgur_client.gallery_search('cat', window='all', sort='time', page=p)
23 for i in range(0,https://gist.github.com/monajalal/e02792e9a5cbced301a8691b7a62836f len(search)):
24 item_count +=1
25 print(search[i].comment_count)
26 if search[i].comment_count > 10 and not search[i].is_album:
27 print(search[i].type)
28 if search[i].type[6:] in image_type:
29 count = 0
30 try:
31 image_file = urllib2.urlopen(search[i].link, timeout = 5)
32 image_file_name = 'images/'+ search[i].id+'.'+search[i].type[6:]
33 output_image = open(image_file_name, 'wb')
34 output_image.write(image_file.read())
35 for post in imgur_client.gallery_item_comments(search[i].id, sort='best'):
36 if count <= 10:
37 count += 1
38 output_image.close()
39 except urllib2.URLError as e:
40 print(e)
41 continue
42 except socket.timeout as e:
43 print(e)
44 continue
45 except socket.error as e:
46 print(e)
47 continue
48 except ImgurClientError as e:
49 print(e)
50 continue
51
52 print item_count
我也经常看到这条线：
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
如何修复错误？ Imgur 中的速率限制错误是否有任何解决方法？所以我创建这个应用程序是为了学术研究而不是商业用途，根据 https://api.imgur.com/#limits 它应该是免费的，但我必须注册我的应用程序才能获得 client_id 相关的东西。如何将我的应用程序设置为非商业应用程序，这样我就不会收到此速率限制错误，或者如果所有类型的应用程序都收到此错误，我应该如何处理？我应该如何设置我的代码以使其每小时仅发出 1250 个请求？
还有我的信用信息：
User Limit: 500
User Remaining: 500
User Reset: 2016-10-18 14:32:41
User Client Limit: 12500
User Client Remaining: 9570
更新：按照答案中的建议使用 sleep(8)，我最终会持续进行下去。对于不同的搜索查询，这发生在不同的页面。我该如何修复代码，以便它在发生这种情况时停止执行？这是更新的相关代码：https://gist.github.com/monajalal/e02792e9a5cbced301a8691b7a62836f
page number is: 157
0
image/jpeg
page number is: 157
0
page number is: 157
0
page number is: 157
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
速率限制是指您点击 API 的频率，而不是您被允许的呼叫次数。为了防止锤击，大多数 API 都有速率限制（例如：每分钟 30 个请求，每 2 秒 1 个）。您的脚本正在尽可能快地发出请求，比限制快数百甚至数千倍。
为了防止您的脚本受到影响，最简单的解决方案是在您的 for 循环中引入一个 sleep。
from time import sleep
for i in range(10000):
print i
sleep(2) # seconds
将休眠时间调整为至少比 API 定义的速率限制长一秒。
The Imgur API uses a credit allocation system to ensure fair distribution of capacity. Each application can allow approximately 1,250 uploads per day or approximately 12,500 requests per day. If the daily limit is hit five times in a month, then the app will be blocked for the rest of the month. The remaining credit limit will be shown with each requests response in the
X-RateLimit-ClientRemainingHTTP header.
所以 12500 个请求/24 小时是每小时 520 个请求或每分钟约 8 个请求。这意味着您的睡眠时间应约为 8 秒。