如何同时添加 spark core 和 mllib 以及 graphx 依赖到 scala 中的 spark 项目 IDE

Question

我已经像这样向我的项目添加了 spark-core 和 spark-graphx 依赖项

group Id: org.apache.spark
Artifact Id: spark-core_2.10
version: 1.6.1

和

group Id: org.apache.spark
Artifact Id: spark-graphx_2.10
version: 1.6.1

进入这些依赖项后，工作区将正确构建，但是当我一开始想添加 mllib 库时，我不知道 spark core 和 graphx 的版本是一致的，我检查了 1.3.0 和 1.1.0，就像之前的两个库一样，通过以下 pom.xml

中的错误

Failure to transfer org.scalamacros:quasiquotes_2.10:jar:2.0.0-M8 from https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer artifact org.scalamacros:quasiquotes_2.10:jar:2.0.0-M8 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org

org.eclipse.aether.transfer.ArtifactTransferException: Failure to transfer org.scalamacros:quasiquotes_2.10:jar:2.0.0-M8 from https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced. Original error: Could not transfer artifact org.scalamacros:quasiquotes_2.10:jar:2.0.0-M8 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException(DefaultUpdateCheckManager.java:238)
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkArtifact(DefaultUpdateCheckManager.java:206)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.gatherDownloads(DefaultArtifactResolver.java:585)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:503)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:205)
    at org.apache.maven.project.DefaultProjectBuilder.resolveDependencies(DefaultProjectBuilder.java:215)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:188)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:119)
    at org.eclipse.m2e.core.internal.embedder.MavenImpl.readMavenProject(MavenImpl.java:636)
    at org.eclipse.m2e.core.internal.project.registry.DefaultMavenDependencyResolver.resolveProjectDependencies(DefaultMavenDependencyResolver.java:63)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryManager.refreshPhase2(ProjectRegistryManager.java:529)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryManager.call(ProjectRegistryManager.java:491)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryManager.call(ProjectRegistryManager.java:1)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryManager.refresh(ProjectRegistryManager.java:495)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryManager.refresh(ProjectRegistryManager.java:350)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryRefreshJob.call(ProjectRegistryRefreshJob.java:89)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryRefreshJob.call(ProjectRegistryRefreshJob.java:1)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
    at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1351)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryRefreshJob.call(ProjectRegistryRefreshJob.java:87)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryRefreshJob.call(ProjectRegistryRefreshJob.java:1)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:176)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
    at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
    at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1355)
    at org.eclipse.m2e.core.internal.project.registry.ProjectRegistryRefreshJob.run(ProjectRegistryRefreshJob.java:81)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)

我还修复了 scala 库到 2.10 版本。如何在我的项目中添加这些依赖项？（我可以在 windows 中添加所有这些，但现在我在 Linux Os 中工作）

Answer 1

我的 Internet 连接很慢，Scala IDE 无法访问依赖项。我已经添加了 1.6.1 版本下的所有依赖项。这些运行现在好了。

如何同时添加 spark core 和 mllib 以及 graphx 依赖到 scala 中的 spark 项目 IDE

how to add spark core and mllib and graphx dependency at the same time to spark project in scala IDE

scala

maven

apache-spark-mllib