通过 RSelenium 单击按钮

Question

我正在尝试使用 Rselarium 和 Rvest 抓取 REI 的评论（Hammocks）。我想点击底部的按钮 x 次，这样我就可以抓取所有评论。我有点迷路了。这是我到目前为止所拥有的。如果您也知道，如何在取景器中预览您正在做的事情（不是屏幕打印）那会很酷。感谢 Stack 社区。

    replicate(100,
          {
remDr$navigate("https://www.amazon.com/Eagles-Nest-Outfitters-DoubleNest-Portable/product-reviews/B00K30GXK8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviewshttps://www.amazon.com/Eagles-Nest-Outfitters-DoubleNest-Portable/product-reviews/B00K30GXK8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews")
webElem <- remDr$findElement("css", "body")
webElem$sendKeysToElement(list(key = "end"))
morereviews <- remDr$findElement(using = 'css selector', ".a-last a")
morereviews$clickElement
Sys.sleep(4)

reviews <- xml2::read_html(remDr$getPageSource()[[1]])%>%
  rvest::html_nodes(".review-text")%>%
  dplyr::data_frame(reviews = .)
})

Answer 1

试试这个：

# Click the Load More button
replicate(100,
          {
            # scroll down
            webElem <- remDr$findElement("css", "body")
            webElem$sendKeysToElement(list(key = "end"))
            # find button
            morereviews <- remDr$findElement(using = 'css selector', "#BVRRContainer div.bv-content-pagination-container button")
            # click button
            morereviews$clickElement()
            # wait
            Sys.sleep(4)
          })

# Scrap the reviews
reviews <- xml2::read_html(remDr$getPageSource()[[1]])%>%
  rvest::html_nodes("#BVRRContainer div.bv-content-summary-body-text") %>%
  rvest::html_text() %>%
  dplyr::data_frame(reviews = .)
reviews

通过 RSelenium 单击按钮

Click Button via RSelenium

r

web-scraping

rselenium

rvest