基于 Windows 8.1 的 Hadoop 2.6.0 构建失败 - Ant BuildException

Question

您好，我正在尝试在 Windows 8.1 上构建 hadoop 2.6.0。不幸的是到目前为止没有运气。

我已经安装了：

jdk1.7.0_71（在用户变量中添加了值为 C:\Program Files\Java\jdk1.7.0_71 的变量 JAVA_HOME）
cygwin64（将其安装目录作为值 D:\cygwin64\bin 添加到系统变量下的 PATH 变量中）
Maven 3.2.5（将其安装目录作为值 D:\maven\bin 添加到系统变量下的 PATH 变量中）
Protocol Buffer 2.5（将其安装目录作为值 D:\protobuf 添加到系统变量下的 PATH 变量中）
Visual Studio 2010

在 Visual Studio 2010 命令提示符（以管理员身份启动）中，我将驱动器更改为 d: 并使用 hadoop src 文件夹作为起点 ("D:\hdp")。

我再次设置 JAVA_HOME 简写并设置平台

set JAVA_HOME=C:\PROGRA~1\Java\jdk1.7.0_71
set Platform=x64

之后，我尝试使用以下 Maven 命令构建 hadoop：

mvn -e -X package -Pdist -DskipTests -Dtar

构建失败后 "Apache Hadoop Common" 说明如下：

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.6.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: protoc failure -> [Help 1]

我稍微更改了 D:\hdp\hadoop-maven-plugins\src\main\java\org\apache\hadoop\maven\plugin\protoc 下的文件 "ProtocMojo.java"，在文件的第 56 行插入以下内容：

protocCommand = "D:\protobuf\protoc.exe";

这有助于进一步构建到 "Apache Hadoop HDFS"，它再次失败，说明如下：

  [INFO] Executing tasks
Build sequence for target(s) `main' is [main]
Complete build sequence is [main, ]
main:
    [mkdir] Skipping D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target\native because it already exists.
     [exec] Current OS is Windows 8.1
     [exec] Executing 'cmake' with arguments:
     [exec] 'D:\hdp\hadoop-hdfs-project\hadoop-hdfs/src/'
     [exec] '-DGENERATED_JAVAH=D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native/javah'
     [exec] '-DJVM_ARCH_DATA_MODEL=64'
     [exec] '-DREQUIRE_LIBWEBHDFS=false'
     [exec] '-DREQUIRE_FUSE=false'
     [exec] '-G'
     [exec] 'Visual Studio 10 Win64'
     [exec]
     [exec] The ' characters around the executable and arguments are
     [exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'cmake' with arguments:
'D:\hdp\hadoop-hdfs-project\hadoop-hdfs/src/'
'-DGENERATED_JAVAH=D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native/javah'
'-DJVM_ARCH_DATA_MODEL=64'
'-DREQUIRE_LIBWEBHDFS=false'
'-DREQUIRE_FUSE=false'
'-G'
'Visual Studio 10 Win64'

The ' characters around the executable and arguments are
not part of the command.
     [exec] CMake Error: Could not create named generator Visual Studio 10 Win64
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [  0.906 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  0.719 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  1.469 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.265 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  1.766 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  5.516 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  1.431 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [  2.119 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  1.969 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:11 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  4.087 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 11.742 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.110 s]
[INFO] Apache Hadoop HDFS ................................. FAILURE [ 11.782 s]
[INFO] Apache Hadoop HttpFS ............................... SKIPPED
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SKIPPED
[INFO] Apache Hadoop HDFS-NFS ............................. SKIPPED
[INFO] Apache Hadoop HDFS Project ......................... SKIPPED
[INFO] hadoop-yarn ........................................ SKIPPED
[INFO] hadoop-yarn-api .................................... SKIPPED
[INFO] hadoop-yarn-common ................................. SKIPPED
[INFO] hadoop-yarn-server ................................. SKIPPED
[INFO] hadoop-yarn-server-common .......................... SKIPPED
[INFO] hadoop-yarn-server-nodemanager ..................... SKIPPED
[INFO] hadoop-yarn-server-web-proxy ....................... SKIPPED
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SKIPPED
[INFO] hadoop-yarn-server-resourcemanager ................. SKIPPED
[INFO] hadoop-yarn-server-tests ........................... SKIPPED
[INFO] hadoop-yarn-client ................................. SKIPPED
[INFO] hadoop-yarn-applications ........................... SKIPPED
[INFO] hadoop-yarn-applications-distributedshell .......... SKIPPED
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SKIPPED
[INFO] hadoop-yarn-site ................................... SKIPPED
[INFO] hadoop-yarn-registry ............................... SKIPPED
[INFO] hadoop-yarn-project ................................ SKIPPED
[INFO] hadoop-mapreduce-client ............................ SKIPPED
[INFO] hadoop-mapreduce-client-core ....................... SKIPPED
[INFO] hadoop-mapreduce-client-common ..................... SKIPPED
[INFO] hadoop-mapreduce-client-shuffle .................... SKIPPED
[INFO] hadoop-mapreduce-client-app ........................ SKIPPED
[INFO] hadoop-mapreduce-client-hs ......................... SKIPPED
[INFO] hadoop-mapreduce-client-jobclient .................. SKIPPED
[INFO] hadoop-mapreduce-client-hs-plugins ................. SKIPPED
[INFO] Apache Hadoop MapReduce Examples ................... SKIPPED
[INFO] hadoop-mapreduce ................................... SKIPPED
[INFO] Apache Hadoop MapReduce Streaming .................. SKIPPED
[INFO] Apache Hadoop Distributed Copy ..................... SKIPPED
[INFO] Apache Hadoop Archives ............................. SKIPPED
[INFO] Apache Hadoop Rumen ................................ SKIPPED
[INFO] Apache Hadoop Gridmix .............................. SKIPPED
[INFO] Apache Hadoop Data Join ............................ SKIPPED
[INFO] Apache Hadoop Ant Tasks ............................ SKIPPED
[INFO] Apache Hadoop Extras ............................... SKIPPED
[INFO] Apache Hadoop Pipes ................................ SKIPPED
[INFO] Apache Hadoop OpenStack support .................... SKIPPED
[INFO] Apache Hadoop Amazon Web Services support .......... SKIPPED
[INFO] Apache Hadoop Client ............................... SKIPPED
[INFO] Apache Hadoop Mini-Cluster ......................... SKIPPED
[INFO] Apache Hadoop Scheduler Load Simulator ............. SKIPPED
[INFO] Apache Hadoop Tools Dist ........................... SKIPPED
[INFO] Apache Hadoop Tools ................................ SKIPPED
[INFO] Apache Hadoop Distribution ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:57 min
[INFO] Finished at: 2015-03-09T17:08:10+01:00
[INFO] Final Memory: 88M/1092M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... @ 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-h
dfs\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project hadoop-hdfs: An Ant BuildExcep
tion has occured: exec returned: 1
around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... @ 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-hdfs\targ
et\antrun\build-main.xml
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:160)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: exec returned: 1
around Ant part ...<exec dir="D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target/native" executable="cmake" failonerror="true">... @ 5:106 in D:\hdp\hadoop-hdfs-project\hadoop-hdfs\targ
et\antrun\build-main.xml
        at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:355)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
        ... 19 more
Caused by: D:\hdp\hadoop-hdfs-project\hadoop-hdfs\target\antrun\build-main.xml:5: exec returned: 1
        at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:646)
        at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:672)
        at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:498)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:390)
        at org.apache.tools.ant.Target.performTasks(Target.java:411)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
        at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:327)
        ... 21 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hadoop-hdfs

这是我卡住的地方，不知道如何解决这个问题。感谢帮助，因为我是 hadoop 和一般建筑的新手。

谢谢！埃尔罗

Answer 1

您是否忘记了 native-win 个人资料？尝试 mvn package -Pdist,native-win -DskipTests -Dtar

我之前在 Windows 上构建过 hadoop，没有更改任何源代码。尝试改回源代码，运行上面的命令，以及 post 更多你在 maven 中遇到的 protoc 错误。

Answer 2

我有同样的错误。通过确保 cmake 不是 cygwin 附带的那个来修复它。出于某种原因，cmake 无法识别 Visual Studio.

基于 Windows 8.1 的 Hadoop 2.6.0 构建失败 - Ant BuildException

Hadoop 2.6.0 build on Windows 8.1 fails - Ant BuildException

hadoop

build

maven

java-7

windows-8.1