R：仅导入和合并特定列

Question

我有一个来自网络的 CSV URL 列表，并将它们合并到一个向量中。现在，我想用 read_csv.

阅读这个列表

示例：

files <- c("csv_link1.csv",
             "csv_link2.csv",
             "csv_link3.csv",
              and so on....) 
data <- map_dfr(files, read_csv)

这没问题。问题是在 CSV 文件中有些列填充了不同的值。因此，例如，在 CSV1 中有列“V1”，它用双精度填充，而在 CSV 中，同一列是“V1”，用字符填充。合并 CSV 无效，因为它们是不同的数据类型。

就我而言，我认为有两种可能性可以解决这个问题。

我只导入某些列，所以我说 read_csv 只读取列（V2 和 V3）而不是 V1

或

我将列合并为与 col_types

我都试过了，但由于语法不正确而失败了。

我试过

data <- map_dfr(files, read_csv(cols_only(the col names)))

但是，这行不通。

如何只导入和合并特定的列？

我的具体例子：

library(data.table)
library(readr)
library(purrr)


files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1819/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1718/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1617/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1516/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1415/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1314/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1213/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1112/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1011/EC.csv")


data <- map_dfr(files, read_csv)

Error: Can't combine `BbAH` <character> and `BbAH` <double>.

所以我的列 BbAH 有不同的数据类型。但我不需要这个专栏。如果我可以选择将在合并运行之前合并到错误中的列，那将会很酷，因为这个不同的数据类型问题。

Answer 1

这个怎么样：

library(data.table)
library(readr)

rbindlist(lapply(files, read_csv, col_types = "character"))

这会将所有列导入为 character，因此您需要在合并后将它们转换为您最初想要的任何内容。

Answer 2

我们可以 select 阅读 csvs 后您需要的列，并使用 map_df 组合它们。

library(tidyverse)
result <- map_df(files, ~read_csv(.x) %>% select(Date,HomeTeam,AwayTeam,FTHG,FTAG,FTR))

Answer 3

由于您只需要这七个变量，您可以使用 fread 读取这些特定变量以避免 BbAH 变量的问题。

library(data.table)
library(dplyr)
library(purrr)

files <- c("https://www.football-data.co.uk/mmz4281/1920/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1819/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1718/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1617/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1516/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1415/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1314/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1213/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1112/EC.csv",
           "https://www.football-data.co.uk/mmz4281/1011/EC.csv")

# Identify columns you need
myColumns = c("Date","Time","HomeTeam","AwayTeam","FTHG","FTAG","FTR")

# Modified function found in 
# takes a filename and a vector of columns as input
fread_allfiles <- function(file, columns){
  x <- fread(file, select = columns) %>% 
    select(everything())   # 
  return(x)
}

df_all <- files %>% 
  map_df(~ fread_allfiles(.,myColumns))

head(df_all)

生成以下格式：

         Date  Time     HomeTeam       AwayTeam FTHG FTAG FTR
1: 03/08/2019 12:30    Stockport     Maidenhead    0    1   A
2: 03/08/2019 15:00    Aldershot          Fylde    1    2   A
3: 03/08/2019 15:00       Barnet         Yeovil    1    0   H
4: 03/08/2019 15:00 Chesterfield Dover Athletic    1    2   A
5: 03/08/2019 15:00      Chorley        Bromley    0    0   D
6: 03/08/2019 15:00  Dag and Red         Woking    0    2   A

然后您可以根据需要重新格式化 Date 和 Time 列。看起来第一个文件有 Time 的任何值？所以剩下的都填成NA

> str(df_all)
Classes ‘data.table’ and 'data.frame':  5429 obs. of  7 variables:
 $ Date    : chr  "03/08/2019" "03/08/2019" "03/08/2019" "03/08/2019" ...
 $ Time    : chr  "12:30" "15:00" "15:00" "15:00" ...
 $ HomeTeam: chr  "Stockport" "Aldershot" "Barnet" "Chesterfield" ...
 $ AwayTeam: chr  "Maidenhead" "Fylde" "Yeovil" "Dover Athletic" ...
 $ FTHG    : int  0 1 1 1 0 0 1 1 2 1 ...
 $ FTAG    : int  1 2 0 2 0 2 0 4 2 3 ...
 $ FTR     : chr  "A" "A" "H" "A" ...
 - attr(*, ".internal.selfref")=<externalptr>

R：仅导入和合并特定列

R: import and merge only specific columns

r

dplyr

data.table

purrr

readr