通过 Python 抓取维基百科图像的问题

Question

我写了一个程序 Python 用于抓取维基百科中查询的第一张图片 Link
类似这张图片的东西：

我的 Python 程序需要以下库：

请求
bs4
html
重新

当我运行我的代码然后我给它一个参数 Returns 定义的错误（'Image-Not-Found'）。请帮助我解决问题。

我的Python程序源代码：

import requests
import bs4
import re
import html

# Create the parser
my_parser = argparse.ArgumentParser(description='Wikipedia Image Grabber')

# Add the arguments
my_parser.add_argument('Phrase',
                       metavar='Phrase',
                       type=str,
                       help='Phrase to Search')

# Execute the parse_args() method
args = my_parser.parse_args()
Phrase = args._get_kwargs()[0][1]
if '.' in Phrase or '-' in Phrase:
    if '.' in Phrase and '-' in Phrase:
        Phrase = str(Phrase).replace('-',' ')
    elif '-' in Phrase and not '.' in Phrase:
        Phrase = str(Phrase).replace('-',' ')

    Phrase = html.escape(Phrase)
request = requests.get('https://fa.wikipedia.org/wiki/Special:Search?search=%s&go=Go&ns0=1' % Phrase).text
parser = bs4.BeautifulSoup(request, 'html.parser')
none_search_finder = parser.find_all('p', attrs = {'class':'mw-search-nonefound'})
if len(none_search_finder)==1:
    print('No-Result')
    exit()
else:
    search_results = parser.find_all('div' , attrs = {'class':'mw-search-result-heading'})
    if len(search_results)==0:
        search_result = parser.find_all('h1', attrs = {'id':'firstHeading'})
        if len(search_result)!=0:
            
            link = 'https://fa.wikipedia.org/wiki/'+str(Phrase)

        else:
            print('Result-Error')
            exit()
    else:

        selected_result = search_results[0]
        regex_exp = r".*<a href=\"(.*)\" title="
        regex_get_uri = re.findall(regex_exp, str(selected_result))
        regex_result = str(regex_get_uri[0])
        link = 'https://fa.wikipedia.org'+regex_result
    
    #---------------
    second_request = requests.get(link)
    second_request_source = second_request.text
    second_request_parser = bs4.BeautifulSoup(second_request_source, 'html.parser')
    image_finder = second_request_parser.find_all('a', attrs = {'class':'image'})
    if len(image_finder) == 0:
        print('No-Image')
        exit()
    else:
        image_finder_e = image_finder[0]
        second_regex = r".*src=\"(.*)\".*decoding=\"async\""
        regex_finder = re.findall(second_regex, str(image_finder_e))
        if len(regex_finder)!=0:
            regexed_uri = str(regex_finder[0])
            img_link = regexed_uri.replace('//','https://')
            print(img_link)
        else:
            print("Image-Not-Found")

Answer 1

你可以在没有正则表达式的情况下做到这一点，你的代码不起作用的原因是在浏览器和响应中 decoding = "async" 位置不一样。

这是一个没有正则表达式的解决方案。

import re
import requests
from bs4 import BeautifulSoup
url = 'https://en.wikipedia.org/wiki/Google'
soup = BeautifulSoup(requests.get(url).text,'html.parser')

imglinks = soup.find_all('a', attrs = {'class':'image'})[0]
for img in imglinks.find_all('img'):
    print(img['src'].replace('//','https://'))

输出：

https://upload.wikimedia.org/wikipedia/commons/thumb/2/2f/Google_2015_logo.svg/196px-Google_2015_logo.svg.png

通过 Python 抓取维基百科图像的问题

Problem To Scraping Wikipedia Images Via Python

python-3.x

regex

web-scraping

argparse

python-requests-html