R + Rvest：从 github 检索文件

Question

抱歉没有提供代表，但如果可以的话，我一开始就不会post。我需要检索在 link

中列出的文件名中包含单词“età”的 excel 文件

https://github.com/apalladi/covid_vaccini_monitoraggio/tree/main/dati

并将它们的文件名存储在向量中。

知道如何实现吗？我正在考虑使用 Rvest，但我愿意接受其他合理的建议。请注意，文件列表需要从 github 页面获取，因为它不是先验的。谢谢！

Answer 1

您应该使用 github API 而不是抓取网站。这样，您可以通过以下操作将文件名和下载链接放入一个漂亮的 two-column 数据框中：

library(httr)
library(dplyr)

req <- GET(paste0("https://api.github.com/repos/", 
                  "apalladi/covid_vaccini_monitoraggio/contents/dati"))

file_list <- content(req)
filenames <- sapply(file_list, function(x) x$name)

file_list <- file_list[grepl("xlsx$", filenames)]

tibble(file = sapply(file_list, function(x) x$name),
       link = sapply(file_list, function(x) x$download_url))
#> # A tibble: 30 x 2
#>    file                         link                                            
#>    <chr>                        <chr>                                           
#>  1 data_iss_età_2021-07-14.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  2 data_iss_età_2021-07-21.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  3 data_iss_età_2021-07-28.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  4 data_iss_età_2021-08-04.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  5 data_iss_età_2021-08-11.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  6 data_iss_età_2021-08-18.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  7 data_iss_età_2021-08-25.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  8 data_iss_età_2021-09-01.xlsx https://raw.githubusercontent.com/apalladi/covi~
#>  9 data_iss_età_2021-09-08.xlsx https://raw.githubusercontent.com/apalladi/covi~
#> 10 data_iss_età_2021-09-15.xlsx https://raw.githubusercontent.com/apalladi/covi~
#> # ... with 20 more rows

^{由 reprex package (v2.0.1)}

创建于 2022-02-01

R + Rvest：从 github 检索文件

R + Rvest: retrieve files from github

r

github

web-scraping

rvest