如何使用多线程在 OSX 上安装 XGBoost
How to install XGBoost on OSX with multi-threading
我正尝试按照指南 here 在我的 mac (osx 10.12.1) 上安装 xgboost，但我 运行 遇到了一些问题。
第一步
Obtain gcc-6.x.x with openmp support by brew install gcc --without-multilib
航站楼
Ben$ brew install gcc --without-multilib
Error: gcc-5.3.0 already installed
To install this version, first `brew unlink gcc`
Ben$ brew unlink gcc
Unlinking /usr/local/Cellar/gcc/5.3.0... 1288 symlinks removed
Ben$ brew install gcc --without-multilib
[26 minutes later]
==> Summary
/usr/local/Cellar/gcc/6.2.0: 1,358 files, 238.3M, built in 26 minutes 20 seconds
第二步
Clone the repository git clone --recursive https://github.com/dmlc/xgboost
航站楼
Ben$ git clone --recursive https://github.com/dmlc/xgboost
Cloning into 'xgboost'...
remote: Counting objects: 18754, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 18754 (delta 1), reused 0 (delta 0), pack-reused 18733
Receiving objects: 100% (18754/18754), 6.23 MiB | 3.74 MiB/s, done.
Resolving deltas: 100% (11347/11347), done.
Checking connectivity... done.
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core) registered for path 'dmlc-core'
Submodule 'rabit' (https://github.com/dmlc/rabit) registered for path 'rabit'
Cloning into '/Users/Ben/xgboost/dmlc-core'...
Cloning into '/Users/Ben/xgboost/rabit'...
Submodule path 'dmlc-core': checked out 'f35f14f30835af238257b979cc1fac3e41ff3291'
Submodule path 'rabit': checked out 'a9a2a69dc1144180a43f7d2d1097264482be7817'
第三步
build using the following commands cd xgboost; cp make/config.mk ./config.mk; make -j4
NOTE: If you use OSX El Capitan, brew installs gcc the latest version gcc-6. So you may need to modify Makefile#L46 and change gcc-5 to gcc-6. After that change gcc-5/g++-5 to gcc-6/g++-6 in make/config.mk then build using the following commands
嗯。不完全确定在这里做什么。我在 xgboost/ 中看到一个名为 "Makefile" 的文件。在里面我看到了很多我不明白的东西，但是这些行看起来相关
Makefile（片段）
# on Mac OS X, force brew gcc-6, since the Xcode c++ fails anyway
# it is useful for pip install compiling-on-the-fly
OS := $(shell uname)
ifeq ($(OS), Darwin)
export CC = $(if $(shell which gcc-6),gcc-6,$(if $(shell which gcc-mp-6), gcc-mp-6, clang))
export CXX = $(if $(shell which g++-6),g++-6,$(if $(shell which g++-mp-6),g++-mp-6, clang++))
endif
将忽略此并继续...
航站楼
Ben$ cd xgboost; cp make/config.mk ./config.mk; make -j4
[Tons of log output. Appears to finish succesfully]
...
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp
Step4（说明here？）
If you would like to use the latest xgboost version and already
compiled xgboost, use library(devtools); install('xgboost/R-package')
to install manually xgboost package (change the path accordingly to
where you compiled xgboost).
新 R 会话
library(devtools)
install("/Users/Ben/xgboost/R-package")
Installing xgboost
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL \
'/Users/Ben/xgboost/R-package' --library='/Library/Frameworks/R.framework/Versions/3.3/Resources/library' --install-tests
* installing *source* package ‘xgboost’ ...
** libs
g++-5 -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../..//include -I../..//dmlc-core/include -I../..//rabit/include -I../../ -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=1 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/opt/llvm/include -fopenmp -fPIC -Wall -mtune=core2 -g -O2 -c xgboost_R.cc -o xgboost_R.o
make: g++-5: No such file or directory
make: *** [xgboost_R.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
Error: Command failed (1)
失败:(我哪里错了？
更新解决方案
截至 2019 年 3 月，R 版本 3.5.3，xgboost 版本 0.82.0.1，自从我的旧答案以来情况发生了变化。以下步骤基于 installation guide here，但略有不同。
brew install cmake 或 brew upgrade cmake（cmake --version 对我显示 3.14.0）brew install gcc 或 brew upgrade gcc（注意安装的 gcc 版本。它应该位于 /usr/local/bin。我得到 gcc 8.3.0 (gcc-8 --version) )cd some/unimportant/directory git clone --recursive https://github.com/dmlc/xgboost cd xgboost/ mkdir buildcd buildCC=gcc-8 CXX=g++-8 cmake .. -DR_LIB=ONmake -j4sudo make install
现在 restart/refresh RStudio 并且应该安装它
在 R 中测试
set.seed(222)
N <- 2*10^5
p <- 350
x <- matrix(rnorm(N * p), ncol = p)
y <- rnorm(N)
system.time(mymodel <- xgboost(
# nthread = 4,
data = x,
label = y,
nrounds = 5,
objective = "reg:linear",
tree_method = "exact",
max_depth = 10,
min_child_weight = 1,
eta = 1,
subsample = 0.66,
colsample_bytree = 0.33
))
# Tested on 2018 MPB, xgboost version 0.82.0.1, multi-threaded version
# nthread = default: 7.4 seconds (elapsed)
# nthread = 1: 24.0 seconds (elapsed)
# nthread = 2: 13.7 seconds (elapsed)
# nthread = 4: 7.5 seconds (elapsed)
旧答案
令人惊奇的是，在 Whosebug 上写下您的问题通常会直接引导您找到答案。花了几个小时之后，我发现我需要更改 ~/.R/Makevars
中的这三行
CC=gcc-5
CXX=g++-5
CXX1X = g++-5
到
CC=gcc-6
CXX=g++-6
CXX1X = g++-6
此外，我最终从 "drat" 存储库
安装了 xgboost
install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")
我的方法是在你的第一个（更新 gcc）和第二个（克隆）步骤之后，不需要构建任何东西（它们适用于其他语言，如 python）
3) 更新 ~/.R/Makevars
https://github.com/dmlc/xgboost/issues/1136
4) 从克隆 xgboost/R-package/configure 将 ac_pkg_openmp 更改为是
https://github.com/dmlc/xgboost/issues/2503
5) 在 xgboost/R-package 中输入 R CMD INSTALL .
http://xgboost.readthedocs.io/en/latest/build.html#r-package-installation
我正尝试按照指南 here 在我的 mac (osx 10.12.1) 上安装 xgboost，但我 运行 遇到了一些问题。
第一步
Obtain gcc-6.x.x with openmp support by
brew install gcc --without-multilib
航站楼
Ben$ brew install gcc --without-multilib
Error: gcc-5.3.0 already installed
To install this version, first `brew unlink gcc`
Ben$ brew unlink gcc
Unlinking /usr/local/Cellar/gcc/5.3.0... 1288 symlinks removed
Ben$ brew install gcc --without-multilib
[26 minutes later]
==> Summary
/usr/local/Cellar/gcc/6.2.0: 1,358 files, 238.3M, built in 26 minutes 20 seconds
第二步
Clone the repository
git clone --recursive https://github.com/dmlc/xgboost
航站楼
Ben$ git clone --recursive https://github.com/dmlc/xgboost
Cloning into 'xgboost'...
remote: Counting objects: 18754, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 18754 (delta 1), reused 0 (delta 0), pack-reused 18733
Receiving objects: 100% (18754/18754), 6.23 MiB | 3.74 MiB/s, done.
Resolving deltas: 100% (11347/11347), done.
Checking connectivity... done.
Submodule 'dmlc-core' (https://github.com/dmlc/dmlc-core) registered for path 'dmlc-core'
Submodule 'rabit' (https://github.com/dmlc/rabit) registered for path 'rabit'
Cloning into '/Users/Ben/xgboost/dmlc-core'...
Cloning into '/Users/Ben/xgboost/rabit'...
Submodule path 'dmlc-core': checked out 'f35f14f30835af238257b979cc1fac3e41ff3291'
Submodule path 'rabit': checked out 'a9a2a69dc1144180a43f7d2d1097264482be7817'
第三步
build using the following commands
cd xgboost; cp make/config.mk ./config.mk; make -j4NOTE: If you use OSX El Capitan, brew installs gcc the latest version gcc-6. So you may need to modify Makefile#L46 and change gcc-5 to gcc-6. After that change gcc-5/g++-5 to gcc-6/g++-6 in make/config.mk then build using the following commands
嗯。不完全确定在这里做什么。我在 xgboost/ 中看到一个名为 "Makefile" 的文件。在里面我看到了很多我不明白的东西，但是这些行看起来相关
Makefile（片段）
# on Mac OS X, force brew gcc-6, since the Xcode c++ fails anyway
# it is useful for pip install compiling-on-the-fly
OS := $(shell uname)
ifeq ($(OS), Darwin)
export CC = $(if $(shell which gcc-6),gcc-6,$(if $(shell which gcc-mp-6), gcc-mp-6, clang))
export CXX = $(if $(shell which g++-6),g++-6,$(if $(shell which g++-mp-6),g++-mp-6, clang++))
endif
将忽略此并继续...
航站楼
Ben$ cd xgboost; cp make/config.mk ./config.mk; make -j4
[Tons of log output. Appears to finish succesfully]
...
g++-6 -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp
Step4（说明here？）
If you would like to use the latest xgboost version and already compiled xgboost, use
library(devtools); install('xgboost/R-package')to install manually xgboost package (change the path accordingly to where you compiled xgboost).
新 R 会话
library(devtools)
install("/Users/Ben/xgboost/R-package")
Installing xgboost
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL \
'/Users/Ben/xgboost/R-package' --library='/Library/Frameworks/R.framework/Versions/3.3/Resources/library' --install-tests
* installing *source* package ‘xgboost’ ...
** libs
g++-5 -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../..//include -I../..//dmlc-core/include -I../..//rabit/include -I../../ -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=1 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/opt/llvm/include -fopenmp -fPIC -Wall -mtune=core2 -g -O2 -c xgboost_R.cc -o xgboost_R.o
make: g++-5: No such file or directory
make: *** [xgboost_R.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/xgboost’
Error: Command failed (1)
失败:(我哪里错了？
更新解决方案
截至 2019 年 3 月，R 版本 3.5.3，xgboost 版本 0.82.0.1，自从我的旧答案以来情况发生了变化。以下步骤基于 installation guide here，但略有不同。
brew install cmake或brew upgrade cmake（cmake --version对我显示 3.14.0）brew install gcc或brew upgrade gcc（注意安装的 gcc 版本。它应该位于/usr/local/bin。我得到 gcc 8.3.0 (gcc-8 --version) )cd some/unimportant/directorygit clone --recursive https://github.com/dmlc/xgboostcd xgboost/mkdir buildcd buildCC=gcc-8 CXX=g++-8 cmake .. -DR_LIB=ONmake -j4sudo make install
现在 restart/refresh RStudio 并且应该安装它
在 R 中测试
set.seed(222)
N <- 2*10^5
p <- 350
x <- matrix(rnorm(N * p), ncol = p)
y <- rnorm(N)
system.time(mymodel <- xgboost(
# nthread = 4,
data = x,
label = y,
nrounds = 5,
objective = "reg:linear",
tree_method = "exact",
max_depth = 10,
min_child_weight = 1,
eta = 1,
subsample = 0.66,
colsample_bytree = 0.33
))
# Tested on 2018 MPB, xgboost version 0.82.0.1, multi-threaded version
# nthread = default: 7.4 seconds (elapsed)
# nthread = 1: 24.0 seconds (elapsed)
# nthread = 2: 13.7 seconds (elapsed)
# nthread = 4: 7.5 seconds (elapsed)
旧答案
令人惊奇的是，在 Whosebug 上写下您的问题通常会直接引导您找到答案。花了几个小时之后，我发现我需要更改 ~/.R/Makevars
CC=gcc-5
CXX=g++-5
CXX1X = g++-5
到
CC=gcc-6
CXX=g++-6
CXX1X = g++-6
此外，我最终从 "drat" 存储库
安装了 xgboostinstall.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")
我的方法是在你的第一个（更新 gcc）和第二个（克隆）步骤之后，不需要构建任何东西（它们适用于其他语言，如 python）
3) 更新 ~/.R/Makevars
https://github.com/dmlc/xgboost/issues/1136
4) 从克隆 xgboost/R-package/configure 将 ac_pkg_openmp 更改为是
https://github.com/dmlc/xgboost/issues/2503
5) 在 xgboost/R-package 中输入 R CMD INSTALL .
http://xgboost.readthedocs.io/en/latest/build.html#r-package-installation