如何将文件的内容逐行传递给 python 中的请求模块

Question

我已经编写了这个脚本来检索网页的内容。

import requests
import bs4

with requests.session() as r:
    r = requests.get("https://www.example.com")
    response = r.text
    print(response)

但是，我在文本文件中有一个 URL 列表。有什么方法可以将此文件的内容直接传递给 requests.get() 而不是手动输入每个内容。

Answer 1

你可以只使用一个循环

假设 file.txt 是您的文件：

with requests.session() as r:
    with open('file.txt') as f:
        for line in f:
            r = requests.get(line)
            response = r.text
            print(response)

Answer 2

把它放在一个循环中。

import requests
import bs4

text_file_name = "list_of_urls.txt"

with requests.session() as session:
    with open(text_file_name) as file:
        for line in file:
            url = line.strip()
            if url:
                resp = session.get(url)
                response = resp.text
                print(response)

注意：您没有使用请求会话对象，因此已修复。

Answer 3

您可以尝试循环所有文件并为每个文件执行 requests.get()

import requests
import bs4

with requests.session() as r:
    with open("urls.txt", "r") as f:
        urls = list(f.readlines())
        for url in urls:
            r = requests.get(url)
            response = r.text
            print("Response for " + url)
            print(response)

Answer 4

import requests
file1 = open('myfile.txt', 'r')
URLS = file1.readlines()

for url in URLS:
    r = requests.get(url)
    response = r.text
    print(response)

这将打印所有 URL 的文本内容

如何将文件的内容逐行传递给 python 中的请求模块

how to pass contents of a file line by line to requests module in python

python

python-requests