为什么保存的文件显示 None 而控制台显示正确的响应?
Why is the file saved showing None while console shows the correct response?
当我尝试将响应保存到文件时,实际响应并未保存,即使它显示在控制台中也是如此。保存在文件中的结果是None。请参阅下面的示例
from concurrent.futures import ThreadPoolExecutor
import requests
#from timer import timer
######### create test file
URLsTest = '''
https://en.wikipedia.org/wiki/NBA
https://en.wikipedia.org/wiki/NFL
'''.strip()
with open('input.txt', 'w') as f:
f.write(URLsTest)
####################
with open('input.txt', 'r') as f:
urls=f.read().split('\n') # url list
def fetch(tt): # received tuple
session, url = tt
print('Processing')
with session.get(url) as response:
print(response.text)
#@timer(1, 5)
def main():
with ThreadPoolExecutor(max_workers=100) as executor:
with requests.Session() as session: # for now, just one session
results = executor.map(fetch, [(session, u) for u in urls]) # tuple list (session, url), each tuple passed to function
executor.shutdown(wait=True)
# write all results to text file
with open('output.txt', 'w') as f2:
for r in results: # tuple (url, html)
f2.write("%s\n" % r)
main()
响应文件 - output.txt
None
None
首先,您可以避免打印 html,因为您正在将该输出保存到文件中。这样你就可以避免使用资源来打印结果。
那么,您的提取不会 return 为 results
获取任何内容。因此,您应该将 print
更改为 return
所以不是打印 return 而是 response.text
# print(response.text)
return response.text
理想的做法是不打印 html,因为您必须将工作或输出保存到一个文件,这样您就无法以原始形状打印整个结果。
当我尝试将响应保存到文件时,实际响应并未保存,即使它显示在控制台中也是如此。保存在文件中的结果是None。请参阅下面的示例
from concurrent.futures import ThreadPoolExecutor
import requests
#from timer import timer
######### create test file
URLsTest = '''
https://en.wikipedia.org/wiki/NBA
https://en.wikipedia.org/wiki/NFL
'''.strip()
with open('input.txt', 'w') as f:
f.write(URLsTest)
####################
with open('input.txt', 'r') as f:
urls=f.read().split('\n') # url list
def fetch(tt): # received tuple
session, url = tt
print('Processing')
with session.get(url) as response:
print(response.text)
#@timer(1, 5)
def main():
with ThreadPoolExecutor(max_workers=100) as executor:
with requests.Session() as session: # for now, just one session
results = executor.map(fetch, [(session, u) for u in urls]) # tuple list (session, url), each tuple passed to function
executor.shutdown(wait=True)
# write all results to text file
with open('output.txt', 'w') as f2:
for r in results: # tuple (url, html)
f2.write("%s\n" % r)
main()
响应文件 - output.txt
None
None
首先,您可以避免打印 html,因为您正在将该输出保存到文件中。这样你就可以避免使用资源来打印结果。
那么,您的提取不会 return 为 results
获取任何内容。因此,您应该将 print
更改为 return
所以不是打印 return 而是 response.text
# print(response.text)
return response.text
理想的做法是不打印 html,因为您必须将工作或输出保存到一个文件,这样您就无法以原始形状打印整个结果。