使用 org.apache.hadoop/* 依赖项进行 sbt 离线编译的问题
Troubles with sbt compiling offline using org.apache.hadoop/* dependencies
使用依赖于 org.apache.hadoop 包的 sbt 进行离线编译时遇到了很多麻烦。
一个简单的build.sbt:
name := "Test"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-api" % "2.2.0"
在线时工作正常，但在 运行 离线时出现以下错误，而包存在于 ivy 缓存中（在 ~/ivy2/cache/org.apache.hadoop/... 下）：
[info] Loading project definition from /home/martin/Dev/S/project
[info] Set current project to Test (in build file:/home/martin/Dev/S/)
[info] Updating {file:/home/martin/Dev/S/}s...
[info] Resolving org.apache.hadoop#hadoop-yarn-api;2.2.0 ...
[warn] Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[warn] module not found: org.apache.hadoop#hadoop-yarn-api;2.2.0
[warn] ==== local: tried
[warn] /home/martin/.ivy2/local/org.apache.hadoop/hadoop-yarn-api/2.2.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.apache.hadoop:hadoop-yarn-api:2.2.0 (/home/martin/Dev/S/build.sbt#L15-16)
[warn] +- test:test_2.10:1.0
sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
...
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] Total time: 3 s, completed Apr 26, 2015 2:46:58 PM
添加以下解析器没有帮助：
resolvers += Resolver.file("Local repo", file(System.getProperty("user.home") + "/.ivy2/cache")) (Resolver.ivyStylePatterns)
它只是添加
[warn] ==== Local repo: tried
[warn] /home/martin/.ivy2/cache/org.apache.hadoop/hadoop-yarn-api/2.2.0/ivys/ivy.xml
文件存在但名为 ivy-2.2。0.xml，而不是 2.2.0/ivys/ivy。xml
所以我尝试添加
resolvers += Resolver.file("Local repo 2", file(System.getProperty("user.home") + "/.ivy2/cache")) ( Patterns("[organisation]/[module]/[artifact]-[revision].[ext]") )
强制使用命名约定，但它会在
下查找
[warn] ==== Local repo 2: tried
[warn] /home/martin/.ivy2/cache/org/apache/hadoop/hadoop-yarn-api/ivy-2.2.0.xml
即使根据 sbt 文档 [organisation] 应该是 org.apache.hadoop 而不是 org/apache/hadoop
所以最后作为最后的手段我添加了一个丑陋的
resolvers += Resolver.file("Local hadoop cache", file(System.getProperty("user.home") + "/.ivy2/cache")) ( Patterns("org.apache.hadoop/[module]/[artifact]-[revision].[ext]") )
它在那里找到了一些东西，但仍然不满意：
[info] Loading project definition from /home/martin/Dev/S/project
[info] Set current project to Test (in build file:/home/martin/Dev/S/)
[info] Updating {file:/home/martin/Dev/S/}s...
[info] Resolving org.apache.hadoop#hadoop-yarn-api;2.2.0 ...
[warn] Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[warn] xml parsing: ivy-2.2.0.xml.original:18:69: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:19:11: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:20:17: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:21:14: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:22:14: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:24:17: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:25:12: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
...
我完全不知道接下来要尝试什么。如果我将 .ivy2/cache/ 目录复制到离线计算机，则构建离线适用于我尝试过的任何其他依赖项。只是一堆 org.apache.hadoop 依赖项导致了这个问题。 .ivy2/cache/org.apache.hadoop 下的结构和文件看起来与其他运行良好的依赖项中的结构和文件相同。
添加
offline := true
也没有帮助。
使用 sbt 0.13.7
谢谢！
似乎是一个 sbt 错误。我在使用 sbt 0.13.8 时遇到了同样的问题，但他们切换到 0.13.9-RC3 并清理了 ~/.ivy2
使用依赖于 org.apache.hadoop 包的 sbt 进行离线编译时遇到了很多麻烦。
一个简单的build.sbt:
name := "Test"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.hadoop" % "hadoop-yarn-api" % "2.2.0"
在线时工作正常，但在 运行 离线时出现以下错误，而包存在于 ivy 缓存中（在 ~/ivy2/cache/org.apache.hadoop/... 下）：
[info] Loading project definition from /home/martin/Dev/S/project
[info] Set current project to Test (in build file:/home/martin/Dev/S/)
[info] Updating {file:/home/martin/Dev/S/}s...
[info] Resolving org.apache.hadoop#hadoop-yarn-api;2.2.0 ...
[warn] Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[warn] module not found: org.apache.hadoop#hadoop-yarn-api;2.2.0
[warn] ==== local: tried
[warn] /home/martin/.ivy2/local/org.apache.hadoop/hadoop-yarn-api/2.2.0/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.apache.hadoop:hadoop-yarn-api:2.2.0 (/home/martin/Dev/S/build.sbt#L15-16)
[warn] +- test:test_2.10:1.0
sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
...
[error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] Total time: 3 s, completed Apr 26, 2015 2:46:58 PM
添加以下解析器没有帮助：
resolvers += Resolver.file("Local repo", file(System.getProperty("user.home") + "/.ivy2/cache")) (Resolver.ivyStylePatterns)
它只是添加
[warn] ==== Local repo: tried
[warn] /home/martin/.ivy2/cache/org.apache.hadoop/hadoop-yarn-api/2.2.0/ivys/ivy.xml
文件存在但名为 ivy-2.2。0.xml，而不是 2.2.0/ivys/ivy。xml
所以我尝试添加
resolvers += Resolver.file("Local repo 2", file(System.getProperty("user.home") + "/.ivy2/cache")) ( Patterns("[organisation]/[module]/[artifact]-[revision].[ext]") )
强制使用命名约定，但它会在
下查找[warn] ==== Local repo 2: tried
[warn] /home/martin/.ivy2/cache/org/apache/hadoop/hadoop-yarn-api/ivy-2.2.0.xml
即使根据 sbt 文档 [organisation] 应该是 org.apache.hadoop 而不是 org/apache/hadoop
所以最后作为最后的手段我添加了一个丑陋的
resolvers += Resolver.file("Local hadoop cache", file(System.getProperty("user.home") + "/.ivy2/cache")) ( Patterns("org.apache.hadoop/[module]/[artifact]-[revision].[ext]") )
它在那里找到了一些东西，但仍然不满意：
[info] Loading project definition from /home/martin/Dev/S/project
[info] Set current project to Test (in build file:/home/martin/Dev/S/)
[info] Updating {file:/home/martin/Dev/S/}s...
[info] Resolving org.apache.hadoop#hadoop-yarn-api;2.2.0 ...
[warn] Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-yarn-api/2.2.0/hadoop-yarn-api-2.2.0.pom
[info] You probably access the destination server through a proxy server that is not well configured.
[warn] xml parsing: ivy-2.2.0.xml.original:18:69: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:19:11: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:20:17: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:21:14: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:22:14: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:24:17: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[warn] xml parsing: ivy-2.2.0.xml.original:25:12: schema_reference.4: Failed to read schema document 'http://maven.apache.org/xsd/maven-4.0.0.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
...
我完全不知道接下来要尝试什么。如果我将 .ivy2/cache/ 目录复制到离线计算机，则构建离线适用于我尝试过的任何其他依赖项。只是一堆 org.apache.hadoop 依赖项导致了这个问题。 .ivy2/cache/org.apache.hadoop 下的结构和文件看起来与其他运行良好的依赖项中的结构和文件相同。
添加
offline := true
也没有帮助。
使用 sbt 0.13.7
谢谢！
似乎是一个 sbt 错误。我在使用 sbt 0.13.8 时遇到了同样的问题，但他们切换到 0.13.9-RC3 并清理了 ~/.ivy2