如何并行化 "make" 可以在多台机器上分配任务的命令
how to parallelize "make" command which can distribute task on multiple machine
我一直在编译“.c / .c++”代码,使用 "make" command.I 在 4 核机器上编译需要 1.5 小时 command.I 还有 10 多台机器我可以用来编译。我知道 "make" 中的“-j”选项,它在指定数量的线程中分发编译。但是“-j”选项只在当前机器上分配线程,而不是在网络连接的其他 10 台机器上。
我们可以使用 MPI 或其他并行编程技术,但我们需要根据并行编程语言重写 "MAKE" 命令实现。
有没有其他方法可以利用其他可用的机器进行编译???
谢谢
是的,有:distcc。
distcc is a program to distribute compilation of C or C++ code across
several machines on a network. distcc should always generate the same
results as a local compile, is simple to install and use, and is often
two or more times faster than a local compile.
Unlike other distributed build systems, distcc does not require all
machines to share a filesystem, have synchronized clocks, or to have
the same libraries or header files installed. Machines can be running
different operating systems, as long as they have compatible binary
formats or cross-compilers.
By default, distcc sends the complete preprocessed source code across
the network for each job, so all it requires of the volunteer machines
is that they be running the distccd daemon, and that they have an
appropriate compiler installed.
他们的关键是你仍然保持你的单一制作,但 gcc 适当地安排文件(运行 预处理器,头文件,......本地)但安排通过网络编译为目标代码。
我过去曾使用过它,它的设置非常简单,而且对您的情况很有帮助。
https://github.com/icecc/icecream
Icecream was created by SUSE based on distcc. Like distcc, Icecream takes compile jobs from a build and distributes it among remote machines allowing a parallel build. But unlike distcc, Icecream uses a central server that dynamically schedules the compile jobs to the fastest free server. This advantage pays off mostly for shared computers, if you're the only user on x machines, you have full control over them.
我一直在编译“.c / .c++”代码,使用 "make" command.I 在 4 核机器上编译需要 1.5 小时 command.I 还有 10 多台机器我可以用来编译。我知道 "make" 中的“-j”选项,它在指定数量的线程中分发编译。但是“-j”选项只在当前机器上分配线程,而不是在网络连接的其他 10 台机器上。
我们可以使用 MPI 或其他并行编程技术,但我们需要根据并行编程语言重写 "MAKE" 命令实现。
有没有其他方法可以利用其他可用的机器进行编译??? 谢谢
是的,有:distcc。
distcc is a program to distribute compilation of C or C++ code across several machines on a network. distcc should always generate the same results as a local compile, is simple to install and use, and is often two or more times faster than a local compile.
Unlike other distributed build systems, distcc does not require all machines to share a filesystem, have synchronized clocks, or to have the same libraries or header files installed. Machines can be running different operating systems, as long as they have compatible binary formats or cross-compilers.
By default, distcc sends the complete preprocessed source code across the network for each job, so all it requires of the volunteer machines is that they be running the distccd daemon, and that they have an appropriate compiler installed.
他们的关键是你仍然保持你的单一制作,但 gcc 适当地安排文件(运行 预处理器,头文件,......本地)但安排通过网络编译为目标代码。
我过去曾使用过它,它的设置非常简单,而且对您的情况很有帮助。
https://github.com/icecc/icecream
Icecream was created by SUSE based on distcc. Like distcc, Icecream takes compile jobs from a build and distributes it among remote machines allowing a parallel build. But unlike distcc, Icecream uses a central server that dynamically schedules the compile jobs to the fastest free server. This advantage pays off mostly for shared computers, if you're the only user on x machines, you have full control over them.