使用 rvest/RSelenium 从下拉菜单中提取链接
Extracting links from a drop down menu using rvest/RSelenium
我正在尝试从下拉框中抓取地名列表。 (下拉框对应下面link(https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l)
中的“Zona”
我可以使用以下方法读入 html 数据:
url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
html_full_page = url %>%
read_html()
但是,我似乎找不到正确的 xpath
或正确的节点。
html_full_page %>%
html_nodes(xpath = '//*[@id="App"]/div[2]/div/div[2]/div[3]/div/div[1]')
我认为节点将对应于以下内容:
re-GeographicSearchNext-linksList-items
但我似乎无法收集这些数据。
我要采集的是下拉菜单的小数据框。
Place Name Properties URL
A bARCALA 69 /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
Arzùa 105 /es/comprar/viviendas/a-coruna-provincia/arzua/l
Barbanza 636 /es/comprar/viviendas/a-coruna-provincia/barbanza/l
从网页复制元素:
> <div class="re-GeographicSearchNext-linksList-items"><a
> class="re-GeographicSearchNext-linkItem" title="A Barcala"
> href="/es/comprar/viviendas/a-coruna-provincia/a-barcala/l"><span
> class="re-GeographicSearchNext-linkItem-literal">A Barcala</span><span
> class="re-GeographicSearchNext-linkItem-count">69</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Arzúa"
> href="/es/comprar/viviendas/a-coruna-provincia/arzua/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Arzúa</span><span
> class="re-GeographicSearchNext-linkItem-count">105</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Barbanza"
> href="/es/comprar/viviendas/a-coruna-provincia/barbanza/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Barbanza</span><span
> class="re-GeographicSearchNext-linkItem-count">636</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Bergantiños"
> href="/es/comprar/viviendas/a-coruna-provincia/bergantinos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Bergantiños</span><span
> class="re-GeographicSearchNext-linkItem-count">581</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de A Coruña"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-a-coruna/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de A
> Coruña</span><span
> class="re-GeographicSearchNext-linkItem-count">3.701</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Betanzos"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-betanzos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Betanzos</span><span
> class="re-GeographicSearchNext-linkItem-count">715</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Ferrol"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-ferrol/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Ferrol</span><span
> class="re-GeographicSearchNext-linkItem-count">3.698</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Santiago"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-santiago/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Santiago</span><span
> class="re-GeographicSearchNext-linkItem-count">1.910</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Eume"
> href="/es/comprar/viviendas/a-coruna-provincia/eume/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Eume</span><span
> class="re-GeographicSearchNext-linkItem-count">282</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Fisterra"
> href="/es/comprar/viviendas/a-coruna-provincia/fisterra/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Fisterra</span><span
> class="re-GeographicSearchNext-linkItem-count">198</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Muros"
> href="/es/comprar/viviendas/a-coruna-provincia/muros/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Muros</span><span
> class="re-GeographicSearchNext-linkItem-count">78</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Noia"
> href="/es/comprar/viviendas/a-coruna-provincia/noia/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Noia</span><span
> class="re-GeographicSearchNext-linkItem-count">192</span></a><a
> class="re-GeographicSearchNext-linkItem" title="O Sar"
> href="/es/comprar/viviendas/a-coruna-provincia/o-sar/l"><span
> class="re-GeographicSearchNext-linkItem-literal">O Sar</span><span
> class="re-GeographicSearchNext-linkItem-count">79</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ordes"
> href="/es/comprar/viviendas/a-coruna-provincia/ordes/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ordes</span><span
> class="re-GeographicSearchNext-linkItem-count">128</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ortegal"
> href="/es/comprar/viviendas/a-coruna-provincia/ortegal/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ortegal</span><span
> class="re-GeographicSearchNext-linkItem-count">248</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Melide"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-melide/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Melide</span><span
> class="re-GeographicSearchNext-linkItem-count">126</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Soneira"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-soneira/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Soneira</span><span
> class="re-GeographicSearchNext-linkItem-count">68</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Xallas"
> href="/es/comprar/viviendas/a-coruna-provincia/xallas/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Xallas</span><span
> class="re-GeographicSearchNext-linkItem-count">20</span></a></div>
我们可以从下拉菜单数据中获取数据,先点击元素,然后提取所需信息,
library(RSelenium)
library(rvest)
rD <- rsDriver(browser="firefox", port=4536L)
remDr <- rD[["client"]]
#navigate
url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
remDr$navigate(url)
#click on Zona
remDr$findElement(using = "xpath",'//*[@id="save-results-panel-trigger"]')$clickElement()
html_full_page = remDr$getPageSource()[[1]] %>% read_html()
names = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>%
html_text()
number = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>% html_nodes('.re-GeographicSearchNext-linkItem-count') %>%
html_text()
link = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>%
html_attr('href')
df = cbind.data.frame(names, number, link)
names number link
1 A Barcala69 69 /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
2 Arzúa105 105 /es/comprar/viviendas/a-coruna-provincia/arzua/l
3 Barbanza636 636 /es/comprar/viviendas/a-coruna-provincia/barbanza/l
4 Bergantiños581 581 /es/comprar/viviendas/a-coruna-provincia/bergantinos/l
我正在尝试从下拉框中抓取地名列表。 (下拉框对应下面link(https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l)
中的“Zona”我可以使用以下方法读入 html 数据:
url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
html_full_page = url %>%
read_html()
但是,我似乎找不到正确的 xpath
或正确的节点。
html_full_page %>%
html_nodes(xpath = '//*[@id="App"]/div[2]/div/div[2]/div[3]/div/div[1]')
我认为节点将对应于以下内容:
re-GeographicSearchNext-linksList-items
但我似乎无法收集这些数据。
我要采集的是下拉菜单的小数据框。
Place Name Properties URL
A bARCALA 69 /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
Arzùa 105 /es/comprar/viviendas/a-coruna-provincia/arzua/l
Barbanza 636 /es/comprar/viviendas/a-coruna-provincia/barbanza/l
从网页复制元素:
> <div class="re-GeographicSearchNext-linksList-items"><a
> class="re-GeographicSearchNext-linkItem" title="A Barcala"
> href="/es/comprar/viviendas/a-coruna-provincia/a-barcala/l"><span
> class="re-GeographicSearchNext-linkItem-literal">A Barcala</span><span
> class="re-GeographicSearchNext-linkItem-count">69</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Arzúa"
> href="/es/comprar/viviendas/a-coruna-provincia/arzua/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Arzúa</span><span
> class="re-GeographicSearchNext-linkItem-count">105</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Barbanza"
> href="/es/comprar/viviendas/a-coruna-provincia/barbanza/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Barbanza</span><span
> class="re-GeographicSearchNext-linkItem-count">636</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Bergantiños"
> href="/es/comprar/viviendas/a-coruna-provincia/bergantinos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Bergantiños</span><span
> class="re-GeographicSearchNext-linkItem-count">581</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de A Coruña"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-a-coruna/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de A
> Coruña</span><span
> class="re-GeographicSearchNext-linkItem-count">3.701</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Betanzos"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-betanzos/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Betanzos</span><span
> class="re-GeographicSearchNext-linkItem-count">715</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Ferrol"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-ferrol/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Ferrol</span><span
> class="re-GeographicSearchNext-linkItem-count">3.698</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Comarca de Santiago"
> href="/es/comprar/viviendas/a-coruna-provincia/comarca-de-santiago/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Comarca de
> Santiago</span><span
> class="re-GeographicSearchNext-linkItem-count">1.910</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Eume"
> href="/es/comprar/viviendas/a-coruna-provincia/eume/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Eume</span><span
> class="re-GeographicSearchNext-linkItem-count">282</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Fisterra"
> href="/es/comprar/viviendas/a-coruna-provincia/fisterra/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Fisterra</span><span
> class="re-GeographicSearchNext-linkItem-count">198</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Muros"
> href="/es/comprar/viviendas/a-coruna-provincia/muros/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Muros</span><span
> class="re-GeographicSearchNext-linkItem-count">78</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Noia"
> href="/es/comprar/viviendas/a-coruna-provincia/noia/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Noia</span><span
> class="re-GeographicSearchNext-linkItem-count">192</span></a><a
> class="re-GeographicSearchNext-linkItem" title="O Sar"
> href="/es/comprar/viviendas/a-coruna-provincia/o-sar/l"><span
> class="re-GeographicSearchNext-linkItem-literal">O Sar</span><span
> class="re-GeographicSearchNext-linkItem-count">79</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ordes"
> href="/es/comprar/viviendas/a-coruna-provincia/ordes/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ordes</span><span
> class="re-GeographicSearchNext-linkItem-count">128</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Ortegal"
> href="/es/comprar/viviendas/a-coruna-provincia/ortegal/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Ortegal</span><span
> class="re-GeographicSearchNext-linkItem-count">248</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Melide"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-melide/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Melide</span><span
> class="re-GeographicSearchNext-linkItem-count">126</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Terra de Soneira"
> href="/es/comprar/viviendas/a-coruna-provincia/terra-de-soneira/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Terra de
> Soneira</span><span
> class="re-GeographicSearchNext-linkItem-count">68</span></a><a
> class="re-GeographicSearchNext-linkItem" title="Xallas"
> href="/es/comprar/viviendas/a-coruna-provincia/xallas/l"><span
> class="re-GeographicSearchNext-linkItem-literal">Xallas</span><span
> class="re-GeographicSearchNext-linkItem-count">20</span></a></div>
我们可以从下拉菜单数据中获取数据,先点击元素,然后提取所需信息,
library(RSelenium)
library(rvest)
rD <- rsDriver(browser="firefox", port=4536L)
remDr <- rD[["client"]]
#navigate
url = 'https://www.fotocasa.es/es/comprar/viviendas/a-coruna-provincia/todas-las-zonas/l'
remDr$navigate(url)
#click on Zona
remDr$findElement(using = "xpath",'//*[@id="save-results-panel-trigger"]')$clickElement()
html_full_page = remDr$getPageSource()[[1]] %>% read_html()
names = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>%
html_text()
number = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>% html_nodes('.re-GeographicSearchNext-linkItem-count') %>%
html_text()
link = html_full_page %>%
html_nodes('.re-GeographicSearchNext-linkItem') %>%
html_attr('href')
df = cbind.data.frame(names, number, link)
names number link
1 A Barcala69 69 /es/comprar/viviendas/a-coruna-provincia/a-barcala/l
2 Arzúa105 105 /es/comprar/viviendas/a-coruna-provincia/arzua/l
3 Barbanza636 636 /es/comprar/viviendas/a-coruna-provincia/barbanza/l
4 Bergantiños581 581 /es/comprar/viviendas/a-coruna-provincia/bergantinos/l