从 R 中的 html 节点列表中提取以 .pdf 结尾的 url

Question

我有一个 url 的列表，每个列表都包含一个指向 pdf 文档的 url。我想使用 R 提取和下载 pdf 文档。这是我到目前为止的代码：

正在从 reliefweb.int

下载数据

#get all the results for the Afghanistan HNO search
result <- GET("https://api.reliefweb.int/v1/reports?appname=rwint-user-0&profile=list&preset=latest&slim=1&query[value]=(primary_country.iso3%3A%22afg%22)%20AND%20ocha_product%3A%22Humanitarian%20Needs%20Overview%22%20AND%20source%3A%22UN%20Office%20for%20the%20Coordination%20of%20Humanitarian%20Affairs%22&query[operator]=AND")

#create a list of all the urls listed in the search page
rawToChar(result$content)
result2<- fromJSON(rawToChar(result$content))
urllist<- result2[["data"]][["fields"]][["url"]]

#Extraxt links to the pdf docs
urlpdf<- lapply(urllist,read_html)

使用此代码，我得到了 html 个节点的列表，但我仍然不知道如何从中提取 .pdf urls。知道我该如何继续或是否有更有效的方法吗？

Answer 1

您似乎在使用 rvest，因此您可以：

library(httr)
library(rvest)
library(jsonlite)

result2<- fromJSON(rawToChar(result$content))
urllist<- result2[["data"]][["fields"]][["url"]]

urlpdf<- lapply(urllist, read_html)

links <- lapply(urlpdf, function(x) html_attr(html_nodes(x, xpath = "//a"), "href"))

pdfs <- lapply(links, function(x) grep("\.pdf$", x, value = TRUE))

这导致：

pdfs
#> [[1]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/afg_humanitarian_needs_overview_2020.pdf"
#> 
#> [[2]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/afg_2019_humanitarian_needs_overview.pdf"
#> 
#> [[3]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/afg_2018_humanitarian_needs_overview_1.pdf"
#> 
#> [[4]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/afg_2017_hno_english.pdf"
#> 
#> [[5]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/afg_2016_hno_final_20151209.pdf"
#> 
#> [[6]]
#> [1] "https://reliefweb.int/sites/reliefweb.int/files/resources/Afghanistan%20HRP%202015%20HNO%20Final%2023Nov2014%20%281%29.pdf"
#> 
#> [[7]]
#> [1] "https://afg.humanitarianresponse.info/system/files/documents/files/Afg_2014HNO_FINALv2_0.pdf"
#> [2] "https://reliefweb.int/sites/reliefweb.int/files/resources/Afg_2014HNO_FINALv2_0.pdf"

从 R 中的 html 节点列表中提取以 .pdf 结尾的 url

extracting url ending in .pdf from a list of html nodes in R

r

html-parsing