使用 Selenium 抓取 React-table

Question

我已经编写了一个代码，用于使用 python selenium 对 HTML React table 进行转义。但它无法捕获 table 中的值（仅 DOM 元素）。这是网站 https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0 这是我的代码：

from selenium import webdriver
dr = webdriver.PhantomJS(r'PATH_TO_PHANTOM/phantomjs-2.1.1-macosx/bin/phantomjs')
dr.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
table = dr.find_elements_by_class_name("rt-tr-group")
for item in table:
    headers = ",".join([th.text for th in item.find_elements_by_class_name("rt-td")])
    print(headers)

它什么都不打印。

感谢您的帮助！

Answer 1

如果您打算使用 and google-chrome you can use the following :

代码块：

driver.get("https://nonfungible.com/market/history/decentraland?filter=saleType%3D&length=10&sort=blockTimestamp%3Ddesc&start=0")
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.rt-tr-group div.rt-td")))])

控制台输出：

['LAND', '115792089237316195423570985008687907834894736851909887012435355206597646221309', '', '', '', '8.462', '20,000 (MANA)', '1 hr ago 7/14/2020, 12:23:58 PM', 'DCLENS', '43469741616618090506261857019714019688977185639939981493731790027408795972144', '', '', '', '.141', '333 (MANA)', '3 hrs ago 7/14/2020, 10:48:46 AM', 'DCLENS', '89838368294314845493020349053893353532486044687731983038462074466093608387295', '', '', '', '.627', '295 (MANA)', '3 hrs ago 7/14/2020, 10:44:24 AM', 'DCLENS', '25527909519625043332149265050323189196974713653740181118935970777552699458072', '', '', '', '.035', '100 (MANA)', '5 hrs ago 7/14/2020, 8:23:03 AM', 'DCLXMAS2019', '349', '', '', '', '.91', '1,251 (MANA)', '7 hrs ago 7/14/2020, 6:11:31 AM', 'LAND', '1361129467683753853853498429727072845789', '', '', '', '7.357', '22,000 (MANA)', '9 hrs ago 7/14/2020, 4:48:01 AM', 'DCLLNCH', '3629', '', '', '', '.164', '200 (MANA)', '10 hrs ago 7/14/2020, 3:04:35 AM', 'LAND', '115792089237316195423570985008687907837276713420356456256678977458620023701615', '', '', '', '9.161', '14,960 (MANA)', '11 hrs ago 7/14/2020, 2:38:29 AM', 'DCLENS', '93012029374970918876221125714905812452590679293106342049263811906231821801921', '', '', '', '.086', '100 (MANA)', '12 hrs ago 7/14/2020, 1:39:25 AM', 'DCLMNSHT2020', '2260', '', '', '', '.138', '0.08 (ETH)', '13 hrs ago 7/14/2020, 1:00:43 AM']

使用 Selenium 抓取 React-table

Scraping a React-table using Selenium

selenium

web-scraping

react-table

webdriverwait

expected-condition