<table> 变成空的，当我试图通过 BeautifulSoup 获取它时

Question

我正在尝试从网站 https://www.kp.ru/best/kazan/abiturient_2018/ivmit/ 解析 table。 Chrome 的 DevTools 显示 table 是：

<div class="t431__table-wapper" data-auto-correct-mobile-width="false"> 
<table class="t431__table " style="">
...
</table>
</div>

但是当我这样做时：

url = r"https://www.kp.ru/best/kazan/abiturient_2018/ivmit/"
r = requests.get(url)
soup = BeautifulSoup(r.text, 'html.parser')
tag = soup.find_all('div', {'class':r't431__table-wapper'})
print(tag)

它returns我喜欢<table>是空的：

[<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>, 
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>,
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>,
<div class="t431__table-wapper" data-auto-correct-mobile-width="false">
<table class="t431__table" style=""></table></div>]

是JavaScript还是什么？如何解决这个问题？

Answer 1

您可以从另一个标签获取该信息

import requests
from bs4 import BeautifulSoup as bs

url = 'https://www.kp.ru/best/kazan/abiturient_2018/ivmit/'
soup = bs(requests.get(url).content, 'lxml')
print(soup.select_one('.t431__data-part2').text)

输出：

<table> 变成空的，当我试图通过 BeautifulSoup 获取它时

<table> becomes empty, when I'm trying to get it via BeautifulSoup

python

parsing

screen-scraping

beautifulsoup

web-scraping