x 多次循环后更改代理

Question

我有一个包含 100 个 URL 的列表。我想每 10 个 URLS 后更改 IP。

假设我有自己的代理，希望在每 10 个 URL 之后使用。我将如何在我的请求中使用该代理 - ?

list = [100URLS items]
proxies ['ip:port','ip:port']
for urls in list:
   try:
       ##request 10 URLS here then it might throw me error. 
   except:
       #After it throws me error, I want to be able to use proxies inside a list something like this and reiterate the same request with a new proxy using requests.

Answer 1

#!/usr/bin/python

import requests

class Proxer:
    proxy = ''
    list = ['http://proxy1','http://proxy2', 'http://pox']
    proxy_count = 0
    page_count = 0

    def proxy_changer(self):
        try:
            if self.proxy_count > 0:
                self.proxy_count = self.proxy_count + 1
            self.proxy = self.list[self.proxy_count]
            return self.proxy
        except:
            print "you are out of proxies"


    def open_site(self, url):
        self.page_count = self.page_count + 1
        try:
            if self.page_count%10:
                self.proxy_changer()
        except:
            pass
        requests.get(url, {'http':self.proxy})


Proxer().open_site('http://google.com')

这是完整的代码。应该在 10 页后使用 open_site('<a href="http://google.com" rel="nofollow">http://google.com</a>') 更改代理一旦你运行没有代理，将返回一个异常。

x 多次循环后更改代理

Change proxy after x many loops

python

syntax

python-requests

proxies