mc.cores > windows 不支持 1

Question

我是 R 编程的新手，我有如下代码，我知道 windows 不支持多核，但我不知道如何更改这部分代码. 有人可以在不使用 mc.cores 功能的情况下向我推荐等效代码吗？

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )
outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

Answer 1

您必须澄清 st_linestring 是什么或做什么，因为您试图将 waydf$geometry$coordinates 的内容传递给它，但没有指定任何参数，例如 st_linestring(waydf$geometry$coordinates[i])

在 Windows 中，您将使用 parLapply 而不是 mclapply。

# original
ll <- parallel::mclapply( waydf$geometry$coordinates , st_linestring,
                         mc.cores =parallel::detectCores() - 1  )

# replace the above with all of the below
library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

编辑因为st_linestring是包sf中的一个函数，导出sf

就足够了

第二次编辑

rpl <- unlist( lapply( waydf$geometry$coordinates , nrow ) ) # row per line
 waydf <- waydf[ rpl > 1 , ]

library(parallel)
cl <- makeCluster(detectCores())
cl <- clusterEvalQ(cl, { library(sf) })  # you need to export packages as well
# cl <- clusterExport(cl, "st_linestring")  # each worker is a new environment, you will need to export variables/functions to
ll <- parallel::parLapply(cl, waydf$geometry$coordinates, function(i) st_linestring)    # if st_linestring takes arguments then st_linestring(x)
stopCluster(cl)

outdf <- sf::st_sf(
line_geometry = sf::st_sfc( ll , crs = epsg ) ,
osm_id = waydf$id
)

Answer 2

如果你只想让这段代码不运行并行，你只需要告诉它使用1个核心，那么它就会在lapply下使用引擎盖。

ll <- parallel::mclapply(waydf$geometry$coordinates, st_linestring, mc.cores = 1)

或者只是将 mclapply 换成 lapply。

ll <- lapply(waydf$geometry$coordinates, st_linestring)

Answer 3

问题是您在所有这些行中都做了 cl <- ...；您不断将该变量重新定义为其他变量。您应该只分配 cl 一次，然后再使用它。

library("parallel")
cl <- makeCluster(detectCores())
clusterEvalQ(cl, { library("sf") })
clusterExport(cl, "st_linestring")
res <- parallel::parLapply(cl, X = waydf$geometry$coordinates, 
                         fun = function(i) st_linestring)
stopCluster(cl)

您通过代码获得的消息 Error in checkCluster(cl): not a valid cluster 是因为在您执行 cl <- clusterEvalQ(cl, { library("sf") }) 之后它不再是 cluster 对象。

mc.cores > windows 不支持 1

mc.cores > 1 is not support on windows

windows

parallel-processing

multicore

r