如何在 CloudFoundry 花园容器中部署文件名大于 242 个字符的 Scala 应用程序

How to deploy a Scala application with filenames larger than 242 characters in CloudFoundry garden container

我一直在尝试使用 sbt 程序集部署一个内置在 fat-jar 中的 scala 应用程序。

容器和 scala 编译生成的 class 名称存在众所周知的问题:https://github.com/milessabin/shapeless/wiki/Shapeless-with-SBT-Assembly-inside-Docker,本质上它们不支持大于 ~242 个字符的文件名,并且将无法启动 scala 应用程序如果它们确实包含那些。问题是有几个库(scalaz、play)包含这些。

scalacOptions in assembly ++= Seq("-Xmax-classfile-name", "73")

还尝试替换用于启动的命令,现在我正在研究使用替代构建包,但其中 none 似乎能够解决问题,因为我什至没有得到到启动命令。

我目前的成绩是:

➜  sample-generator git:(master) ✗ cf push sample-gen -p ./target/sample-generator-assembly-201608111441.jar -b java_buildpack  --health-check-type=none -c "sleep(100)"                                          
Creating app sample-gen in org pcfdev-org / space pcfdev-space as user...
OK

Using route sample-gen.local.pcfdev.io
Binding sample-gen.local.pcfdev.io to sample-gen...
OK

Uploading sample-gen...
Uploading app files from: /tmp/unzipped-app826671513
Uploading 120.3M, 38143 files
Done uploading               
OK

Starting app sample-gen in org pcfdev-org / space pcfdev-space as user...
Downloading java_buildpack...
Downloaded java_buildpack
Creating container
Successfully created container
Downloading app package...

FAILED
Error restarting application: StagingError

TIP: use 'cf logs sample-gen --recent' for more information

由于 cf 日志对我来说真的 return 没有任何用处,我必须登录 PCFDev 并通过 /var/vcap/data/sys/log/garden/garden.stdout.log:

获取日志
{"timestamp":"1470918381.941401958","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/app","error":"error streaming in: exit status 2. Output: tar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply8$$anonfun$$$$f6e7f12408688f578bcac985aee12$$$$6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply7$$anonfun$$$$d23ff5fc3821e643c952318248bfc6$$$$5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply$$anonfun$apply$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply7$$anonfun$$$$$a8626ce150a144689738daf9754d5e7$$$$5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2.class: Cannot open: File name too long\ntar: play/core/routing/GeneratedRouter$$anonfun$call$$anonfun$apply3$$anonfun$apply4$$anonfun$apply5$$anonfun$apply6$$anonfun$apply7$$anonfun$apply8$$anonfun$apply9$$anonfun$apply0$$anonfun$apply1$$anonfun$apply2$$anonfun$apply3.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$tupled.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: scalaz/syntax/ApplicativeBuilder$ApplicativeBuilder3$ApplicativeBuilder4$ApplicativeBuilder5$ApplicativeBuilder6$ApplicativeBuilder7$ApplicativeBuilder8$ApplicativeBuilder9$ApplicativeBuilder10$ApplicativeBuilder11$ApplicativeBuilder12$$anonfun$apply.class: Cannot open: File name too long\ntar: Exiting with failure status due to previous errors\n","handle":"a1ef2606-ead0-4cb1-b0fd-d464d64b8bc6-ff54adf06a8342f59eedfb022a9e9527","session":"11.7615","user":"vcap"}}

对于 Java/Scala 用户在应用程序暂存期间仍在为长嵌套文件名中断而苦恼或苦苦挣扎,有一个更简单的解决方案,不需要自定义构建包。

使用Java class路径清单指定附加的classes捆绑在其他jar中,然后推送主应用程序,而不是将所有内容打包为一个大jar文件。最终结构应该是一个包含多个内部 jar 的顶级 Jar 文件,所有内部 jar 都由主要 classpath 清单条目引用。

1) 使用第三方库或依赖项作为单独的 jar 在其他文件夹(如 lib/..)中创建应用程序位

示例结构:

com.mycompany.foo.class
com.mycompany.bar.class
com.mycompany.xyz.class
lib/myDepedencyA.jar
lib/myDepedencyB.jar
lib/myDepedencyC.jar
.....
lib/myDepedencyZ.jar

2) 创建一个 class 路径清单条目,在自定义清单中引用这些额外的 jar(比如说我的清单)。

Manifest-Version: 1.0
Implementation-Vendor: test
Implementation-Title: Test
Implementation-Version: 1.0
Implementation-Vendor-Id: test
Specification-Vendor: test
Specification-Title: TestApp
Specification-Version: 1.0
Main-Class: TestService
Class-Path:  lib/akka-actor_2.10-2.1.4.jar
  lib/casbah-commons_2.10-2.7.1.jar
  lib/casbah-core_2.10-2.7.1.jar
  lib/casbah-gridfs_2.10-2.7.0.jar
  lib/casbah-query_2.10-2.7.1.jar
  lib/config-1.0.0.jar
  lib/joda-convert-1.2.jar
  lib/joda-time-2.3.jar
  lib/json4s-ast_2.10-3.2.9.jar
  lib/json4s-core_2.10-3.2.9.jar
  lib/myDepedencyA.jar
  lib/myDepedencyB.jar
  lib/myDepedencyC.jar
  .....
  lib/myDepedencyZ.jar

3) 在明确指定清单文件的同时创建最终的 jar。

jar cvfm MyApp.jar mymanifest -C foo/ .

说明:使用清单并在包含 classes 之前转到目录 foo。另外,参考 https://docs.oracle.com/javase/tutorial/deployment/jar/downman.html 然后,jar 将捆绑 appln classes 和第 3 方库,但没有每一个有问题的长 class 名称。 推送前先检查manifest内容确定一下。

4) 当应用程序推送发生时,只有顶级应用程序 classes 会被打开,而不会从 lib 文件夹中爆炸第 3 方罐子。 java buildpack 会在运行时自动将 classpath 清单中指定的 jar 添加到服务器 classpath 中。

start_command: $PWD/.java-buildpack/open_jdk_jre/bin/java -cp $PWD/.:$PWD/lib/akka-actor_2.10-2.1.4.jar:$PWD/lib/casbah-commons_2 .10-2.7.1.jar:$PWD/lib/casbah-core_2.10-2.7.1.jar:$PWD/lib/casbah-gridfs_2.10-2.7.0.jar:$PWD/lib/casbah-query_2.10-2.7.1.jar:$PWD/lib/config-1.0.0.jar:$PWD/lib/joda-convert-1.2.jar:$PWD/lib/joda-time-2.3.jar:$PWD/lib/json4s-ast_2.10-3.2.9.jar:$PWD/lib/json4s-core_2.10-3.2.9.jar:$PWD/lib/json4s-native_2.10-3.2.9.jar:$PWD/lib/mimepull-1.9.4.jar:$PWD/lib/mongo-java-driver-2.12.1.jar:$PWD/lib/nscala-time_2.10-1.0.0.jar :$PWD/lib/paranamer-2.6.jar:$PWD/lib/parboiled-core-1.1.6.jar:$PWD/lib/parboiled-scala_2.10-1.1.6.jar:$PWD/lib/salat-core_2.10-1.9.8.jar:$PWD/lib/salat-util_2.10-1.9.8.jar:$PWD/lib/sbinary_2.10-0.4.2.jar:$PWD/lib/sbt-0.13.1.jar:$PWD/lib/sbt-assembly-0.11.2.jar:$PWD/lib/sbt-idea-1.6.0.jar:$PWD/lib/scala-compiler-2.10 .4.jar:$PWD/lib/scala-library-2.10.4.jar:$PWD/lib/scala-reflect-2.10.4.jar:$PWD/lib/scalap-2.10.4.jar:$PWD/lib/shapeless_2.10-1.2.4.jar:$PWD/lib/slf4j-api-1.7.2.jar:$PWD/lib/spray-can-1.1.1.jar:$ PWD/lib/spray-http-1.1.1.jar:$PWD/lib/spray-httpx-1.1.1.jar:$PWD/lib/spray-io-1.1.1.jar:$PWD/lib/spray-json_2 .10-1.2.5.jar:$PWD/lib/spray-routing-1.1.1.jar:$PWD/lib/spray-util-1.1.1.jar:$PWD/lib/task-system-0.13. 1.jar:$PWD/lib/tasks-0.13.1.jar:$PWD/lib/test-agent-0.13.1.jar:$PWD/lib/test-interface-1.0.jar:$ PWD/lib/testing-0.13.1.jar:$PWD/lib/tracking-0.13.1.jar -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh -Xmx1536M -Xms1536M -XX:MaxPermSize=209715K -XX:PermSize=209715K -Xss1M TestService

以防万一这可能仍然对任何人有帮助,我们已经设法使用 sbt-native-packager 而不是 sbt-assembly 的解决方案。

使用 sbt-native-packager 和 universal:packageBin 命令可以生成一个 zip,包含与 sbt stage 生成的相同内容。 jar 保持独立(未分解),避免了 sbt-assembly 导致的文件大小产生的编译问题。之后 java_buildpack 能够 运行 该应用程序而不会出现问题:

cf push sample-app -p ./target/universal/stage -b java_buildpack --health-check-type=none

cf push sample-app -p ./target/universal/sample-app-201608160251.zip -b java_buildpack --health-check-type=none