如何从文本文件中读取 url 列表并将所有响应保存到一个文本文件中？

Question

我在文本文件中有一个 url 的列表。我想将文本文件中的 urls 馈送到我的代码脚本中，而不是将每个 url 粘贴到脚本中，但不确定如何操作。我还想将所有回复保存到一个文本文件中。谢谢

import requests
from datetime import datetime

def run():

    url = "https://en.wikipedia.org/wiki/NBA"

    payload = {}
    headers= {}

    response = requests.request("POST", url, headers=headers, data = payload)

    print(response.text)

startTime = datetime.now()
run()
print(datetime.now() - startTime)

文件：Wiki.text

https://en.wikipedia.org/wiki/NBA
https://en.wikipedia.org/wiki/NFL
https://en.wikipedia.org/wiki/MLB
https://en.wikipedia.org/wiki/NHL

Answer 1

你可以这样做：

import requests
list_of_pages = open('list_of_links.txt', 'r+')
save = open('output.txt', 'a+')
save.truncate(0)
for page_link in list_of_pages:
    url = page_link
    payload = {}
    headers = {}
    response = requests.request("POST", url, headers=headers, data=payload)
    save.write(str(response) + "\n")

Answer 2

试试这个，它 returns Wiki.txt 中每个页面的代码以及获取它所花费的时间：

import requests
from datetime import datetime
def run():
    links_file = open('Wiki.txt', 'r')
    outputs_file = open('outputs.txt', 'a')
    for link in links_file.readlines():  # run for ever line in file
            payload = {}
            headers = {}

            response = requests.request("POST", link, headers=headers, data=payload)

            outputs_file.write(response.text + '\n')
    links_file.close()
    outputs_file.close()

startTime = datetime.now()
run()
print(datetime.now() - startTime)

基本上您需要做的就是为文件中的每一行添加一个循环，并将生成的代码添加到另一个文本文件中。

如何从文本文件中读取 url 列表并将所有响应保存到一个文本文件中？

How can I read a list of urls from a text file and save all the responses into one text file?

python

python-3.x

url

python-requests

readline