从 url 下载 .html 文件时出现超时错误
Timeout error when downloading .html files from urls
从 url 下载 html 页面时出现以下错误。
Error: raise URLError(err) urllib2.URLError: <urlopen error [Errno
10060] A connection attempt failed because the connected party did not
properly respond after a period of time or established connection
failed because connected host has failed to respond>
代码:
import urllib2
hdr = {'User-Agent': 'Mozilla/5.0'}
for i,site in enumerate(urls[index]):
print (site)
req = urllib2.Request(site, headers=hdr)
page = urllib2.build_opener(urllib2.HTTPCookieProcessor).open(req)
page_content = page.read()
with open(path_current+'/'+str(i)+'.html', 'w') as fid:
fid.write(page_content)
我认为这可能是由于某些代理设置或更改超时造成的,但我不确定。请帮助,我手动检查了网址似乎可以正常打开。
嗯,由于大多数时候您不会遇到这种情况,我可以推断您的网络可能很慢。
尝试按以下方式设置超时:
req = urllib2.Request(site, headers=hdr)
timeout_in_sec = 360
page = urllib2.build_opener(urllib2.HTTPCookieProcessor).open(req, timeout=timeout_in_sec)
page_content = page.read()
从 url 下载 html 页面时出现以下错误。
Error: raise URLError(err) urllib2.URLError: <urlopen error [Errno
10060] A connection attempt failed because the connected party did not
properly respond after a period of time or established connection
failed because connected host has failed to respond>
代码:
import urllib2
hdr = {'User-Agent': 'Mozilla/5.0'}
for i,site in enumerate(urls[index]):
print (site)
req = urllib2.Request(site, headers=hdr)
page = urllib2.build_opener(urllib2.HTTPCookieProcessor).open(req)
page_content = page.read()
with open(path_current+'/'+str(i)+'.html', 'w') as fid:
fid.write(page_content)
我认为这可能是由于某些代理设置或更改超时造成的,但我不确定。请帮助,我手动检查了网址似乎可以正常打开。
嗯,由于大多数时候您不会遇到这种情况,我可以推断您的网络可能很慢。 尝试按以下方式设置超时:
req = urllib2.Request(site, headers=hdr)
timeout_in_sec = 360
page = urllib2.build_opener(urllib2.HTTPCookieProcessor).open(req, timeout=timeout_in_sec)
page_content = page.read()