使用 rvest/RSelenium 从下拉菜单中提取链接

Extracting links from a drop down menu using rvest/RSelenium

我正在尝试从下拉框中抓取地名列表。 (下拉框对应下面link(https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l)

中的“Zona

我可以使用以下方法读入 html 数据:

url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
html_full_page = url %>% 
  read_html()

但是,我似乎找不到正确的 xpath 或正确的节点。

html_full_page %>% 
  html_nodes(xpath = '//*[@id="App"]/div[2]/div/div[2]/div[3]/div/div[1]')

我认为节点将对应于以下内容:

re-GeographicSearchNext-linksList-items

但我似乎无法收集这些数据。

我要采集的是下拉菜单的小数据框。

Place Name    Properties    URL
A bARCALA        69         /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
Arzùa           105         /es/comprar/viviendas/a-coruna-provincia/arzua/l
Barbanza        636         /es/comprar/viviendas/a-coruna-provincia/barbanza/l

从网页复制元素:

> <div class="re-GeographicSearchNext-linksList-items"><a
> class="re-GeographicSearchNext-linkItem" title="A Barcala"
> href="/es/comprar/viviendas/a-coruna-provincia/a-barcala/l"><span
> class="re-GeographicSearchNext-linkItem-literal">A Barcala</span><span
> class="re-GeographicSearchNext-linkItem-count">69</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Arzúa"
> href="/es/comprar/viviendas/a-coruna-provincia/arzua/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Arzúa</span><span
> class="re-GeographicSearchNext-linkItem-count">105</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Barbanza"
> href="/es/comprar/viviendas/a-coruna-provincia/barbanza/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Barbanza</span><span
> class="re-GeographicSearchNext-linkItem-count">636</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Bergantiños"
> href="/es/comprar/viviendas/a-coruna-provincia/bergantinos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Bergantiños</span><span
> class="re-GeographicSearchNext-linkItem-count">581</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de A Coruña"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-a-coruna/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de A
> Coruña</span><span
> class="re-GeographicSearchNext-linkItem-count">3.701</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Betanzos"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-betanzos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Betanzos</span><span
> class="re-GeographicSearchNext-linkItem-count">715</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Ferrol"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-ferrol/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Ferrol</span><span
> class="re-GeographicSearchNext-linkItem-count">3.698</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Santiago"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-santiago/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Santiago</span><span
> class="re-GeographicSearchNext-linkItem-count">1.910</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Eume"
> href="/es/comprar/viviendas/a-coruna-provincia/eume/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Eume</span><span
> class="re-GeographicSearchNext-linkItem-count">282</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Fisterra"
> href="/es/comprar/viviendas/a-coruna-provincia/fisterra/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Fisterra</span><span
> class="re-GeographicSearchNext-linkItem-count">198</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Muros"
> href="/es/comprar/viviendas/a-coruna-provincia/muros/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Muros</span><span
> class="re-GeographicSearchNext-linkItem-count">78</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Noia"
> href="/es/comprar/viviendas/a-coruna-provincia/noia/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Noia</span><span
> class="re-GeographicSearchNext-linkItem-count">192</span></a><a
> class="re-GeographicSearchNext-linkItem" title="O Sar"
> href="/es/comprar/viviendas/a-coruna-provincia/o-sar/l"><span
> class="re-GeographicSearchNext-linkItem-literal">O Sar</span><span
> class="re-GeographicSearchNext-linkItem-count">79</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ordes"
> href="/es/comprar/viviendas/a-coruna-provincia/ordes/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ordes</span><span
> class="re-GeographicSearchNext-linkItem-count">128</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ortegal"
> href="/es/comprar/viviendas/a-coruna-provincia/ortegal/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ortegal</span><span
> class="re-GeographicSearchNext-linkItem-count">248</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Melide"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-melide/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Melide</span><span
> class="re-GeographicSearchNext-linkItem-count">126</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Soneira"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-soneira/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Soneira</span><span
> class="re-GeographicSearchNext-linkItem-count">68</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Xallas"
> href="/es/comprar/viviendas/a-coruna-provincia/xallas/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Xallas</span><span
> class="re-GeographicSearchNext-linkItem-count">20</span></a></div>

我们可以从下拉菜单数据中获取数据,先点击元素,然后提取所需信息,

library(RSelenium)
library(rvest)
rD <- rsDriver(browser="firefox", port=4536L)
remDr <- rD[["client"]]
#navigate 
url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
remDr$navigate(url)
#click on Zona
remDr$findElement(using = "xpath",'//*[@id="save-results-panel-trigger"]')$clickElement()


html_full_page = remDr$getPageSource()[[1]] %>% read_html() 

names = html_full_page %>% 
  html_nodes('.re-GeographicSearchNext-linkItem') %>% 
  html_text()
  
number = html_full_page %>% 
  html_nodes('.re-GeographicSearchNext-linkItem') %>% html_nodes('.re-GeographicSearchNext-linkItem-count') %>% 
  html_text()

link  = html_full_page %>% 
  html_nodes('.re-GeographicSearchNext-linkItem') %>% 
  html_attr('href')

df = cbind.data.frame(names, number, link)

                      names number                                                           link
1               A Barcala69     69           /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
2                  Arzúa105    105               /es/comprar/viviendas/a-coruna-provincia/arzua/l
3               Barbanza636    636            /es/comprar/viviendas/a-coruna-provincia/barbanza/l
4            Bergantiños581    581         /es/comprar/viviendas/a-coruna-provincia/bergantinos/l