在 Spring Yarn 应用程序中使用本地文件

Question

我需要在 yarn 容器上部署 war 文件，因为我正在使用 Spring 批处理创建一个使用嵌入式码头服务器部署 war 文件的 yarn 应用程序，当我使用自定义句柄来处理请求，它工作正常，应用程序部署在 yarn 上，但是当我使用 war 文件作为处理程序时，它不起作用我得到一个错误

java.io.FileNotFoundException: /root/jettywebapps/webapps/Login.war

但文件实际上存在于给定位置我不知道如何从 yarn 容器访问 war 文件

@OnContainerStart
    public void publicVoidNoArgsMethod() throws Exception {
        String jetty_home="/root/jettywebapps";
        Server server = new Server(9090);
        WebAppContext webapp = new WebAppContext();
        webapp.setContextPath("/Login");
        webapp.setWar(jetty_home+"/webapps/Login.war");
        server.setHandler(webapp);
        server.start();
        server.join();
    }

这是堆栈跟踪

2015-04-17 06:05:14.972] boot - 26920  INFO [main] --- ContainerLauncherRunner: Running YarnContainer with parameters []
[2015-04-17 06:05:14.972] boot - 26920  INFO [main] --- ContainerLauncherRunner: Container requested that we wait state, setting up latch
[2015-04-17 06:05:14.975] boot - 26920  INFO [main] --- DefaultYarnContainer: Processing 1 @YarnComponent handlers
[2015-04-17 06:05:15.038] boot - 26920  INFO [main] --- Server: jetty-8.0.4.v20111024
[2015-04-17 06:05:15.080] boot - 26920  WARN [main] --- WebInfConfiguration: Web application not found /root/jettywebapps/webapps/Login.war
[2015-04-17 06:05:15.081] boot - 26920  WARN [main] --- WebAppContext: Failed startup of context o.e.j.w.WebAppContext{/Login,null},/root/jettywebapps/webapps/Login.war
java.io.FileNotFoundException: /root/jettywebapps/webapps/Login.war
        at org.eclipse.jetty.webapp.WebInfConfiguration.unpack(WebInfConfiguration.java:479)
        at org.eclipse.jetty.webapp.WebInfConfiguration.preConfigure(WebInfConfiguration.java:52)
        at org.eclipse.jetty.webapp.WebAppContext.preConfigure(WebAppContext.java:416)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:452)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:89)
        at org.eclipse.jetty.server.Server.doStart(Server.java:262)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:59)
        at hello.container.HelloPojo.publicVoidNoArgsMethod(HelloPojo.java:40)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.springframework.expression.spel.support.ReflectiveMethodExecutor.execute(ReflectiveMethodExecutor.java:112)
        at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:129)
        at org.springframework.expression.spel.ast.MethodReference.access[=12=]0(MethodReference.java:49)
        at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:342)
        at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:88)
        at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:131)
        at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:330)
        at org.springframework.yarn.support.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:126)
        at org.springframework.yarn.container.ContainerMethodInvokerHelper.processInternal(ContainerMethodInvokerHelper.java:229)
        at org.springframework.yarn.container.ContainerMethodInvokerHelper.process(ContainerMethodInvokerHelper.java:115)
        at org.springframework.yarn.container.MethodInvokingYarnContainerRuntimeProcessor.process(MethodInvokingYarnContainerRuntimeProcessor.java:51)
        at org.springframework.yarn.container.ContainerHandler.handle(ContainerHandler.java:99)
        at org.springframework.yarn.container.DefaultYarnContainer.getContainerHandlerResults(DefaultYarnContainer.java:174)
        at org.springframework.yarn.container.DefaultYarnContainer.runInternal(DefaultYarnContainer.java:77)

请帮忙谢谢

Answer 1

容器可以使用以下代码获取本地文件或 hdfs 文件：

Configuration conf = new Configuration();
FileSystem localFS = FileSystem.get(URI.create("file://localhost"), conf);
OutputStream outATXT = localFS.create(new Path("/home/walterchen/a.txt"));

或

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create("hdfs://localhost:9000"), conf);
OutputStream out = fs.create(new Path("/home/a.txt"));

在 Spring Yarn 应用程序中使用本地文件

Using Local File in Spring Yarn application

spring-boot

hadoop-yarn