映射目录时，sbt-native-packager 失败并显示 "Too many open files in system"

Question

我有一个基于 sbt 的项目。通用打包插件已启用

.enablePlugins(JavaAppPackaging)

其他目录映射如下：

mappings in Universal ++= directory("public")

我尝试在多台机器上使用 dist 命令构建包（运行 OS X 和 Linux）它总是失败以下痕迹：

    Could not create file [...omitted...]/target/streams/compile/$global/$global/discoveredMainClasses/datajava.io.IOException: Too many open files in system
            at sbt.ErrorHandling$.translate(ErrorHandling.scala:10)
            at sbt.IO$.touch(IO.scala:196)
            at sbt.std.Streams$$anon$$anon.make(Streams.scala:129)
            at sbt.std.Streams$$anon$$anon.binary(Streams.scala:116)
            at sbt.SessionVar$$anonfun$persist.apply(SessionVar.scala:27)
            at sbt.SessionVar$$anonfun$persist.apply(SessionVar.scala:26)
            at sbt.std.Streams$class.use(Streams.scala:75)
            at sbt.std.Streams$$anon.use(Streams.scala:100)
            at sbt.SessionVar$.persist(SessionVar.scala:26)
            at sbt.SessionVar$.persistAndSet(SessionVar.scala:21)
            at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$$anonfun$apply.apply(Project.scala:563)
            at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$$anonfun$apply.apply(Project.scala:563)
            at sbt.SessionVar$$anonfun$$anonfun$apply.apply(SessionVar.scala:40)
            at sbt.SessionVar$$anonfun$$anonfun$apply.apply(SessionVar.scala:40)
            at scala.Function$$anonfun$chain$$anonfun$apply.apply(Function.scala:24)
            at scala.Function$$anonfun$chain$$anonfun$apply.apply(Function.scala:24)
            at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:51)
            at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:60)
            at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:47)
            at scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:138)
            at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:105)
            at scala.Function$$anonfun$chain.apply(Function.scala:24)
            at sbt.EvaluateTask$.applyResults(EvaluateTask.scala:387)
            at sbt.EvaluateTask$.liftedTree1(EvaluateTask.scala:361)
            at sbt.EvaluateTask$.run(EvaluateTask.scala:358)
            at sbt.EvaluateTask$.runTask(EvaluateTask.scala:378)
            at sbt.Aggregation$$anonfun.apply(Aggregation.scala:64)
            at sbt.Aggregation$$anonfun.apply(Aggregation.scala:62)
            at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:314)
            at sbt.Aggregation$.timedRun(Aggregation.scala:62)
            at sbt.Aggregation$.runTasks(Aggregation.scala:71)
            at sbt.Aggregation$$anonfun$applyTasks.apply(Aggregation.scala:32)
            at sbt.Aggregation$$anonfun$applyTasks.apply(Aggregation.scala:31)
            at sbt.Command$$anonfun$applyEffect$$anonfun$apply.apply(Command.scala:61)
            at sbt.Command$$anonfun$applyEffect$$anonfun$apply.apply(Command.scala:61)
            at sbt.Aggregation$$anonfun$evaluatingParser$$anonfun$apply.apply(Aggregation.scala:153)
            at sbt.Aggregation$$anonfun$evaluatingParser$$anonfun$apply.apply(Aggregation.scala:152)
            at sbt.Act$$anonfun$sbt$Act$$actParser0$$anonfun$sbt$Act$$anonfun$$evaluate$$anonfun$apply.apply(Act.scala:244)
            at sbt.Act$$anonfun$sbt$Act$$actParser0$$anonfun$sbt$Act$$anonfun$$evaluate$$anonfun$apply.apply(Act.scala:241)
            at sbt.Command$.process(Command.scala:93)
            at sbt.MainLoop$$anonfun$$anonfun$apply.apply(MainLoop.scala:96)
            at sbt.MainLoop$$anonfun$$anonfun$apply.apply(MainLoop.scala:96)
            at sbt.State$$anon.process(State.scala:184)
            at sbt.MainLoop$$anonfun.apply(MainLoop.scala:96)
            at sbt.MainLoop$$anonfun.apply(MainLoop.scala:96)
            at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
            at sbt.MainLoop$.next(MainLoop.scala:96)
            at sbt.MainLoop$.run(MainLoop.scala:89)
            at sbt.MainLoop$$anonfun$runWithNewLog.apply(MainLoop.scala:68)
            at sbt.MainLoop$$anonfun$runWithNewLog.apply(MainLoop.scala:63)
            at sbt.Using.apply(Using.scala:24)
            at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
            at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
            at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
            at sbt.MainLoop$.runLogged(MainLoop.scala:22)
            at sbt.StandardMain$.runManaged(Main.scala:54)
            at sbt.xMain.run(Main.scala:29)
            at xsbt.boot.Launch$$anonfun$run.apply(Launch.scala:109)
            at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
            at xsbt.boot.Launch$.run(Launch.scala:109)
            at xsbt.boot.Launch$$anonfun$apply.apply(Launch.scala:35)
            at xsbt.boot.Launch$.launch(Launch.scala:117)
            at xsbt.boot.Launch$.apply(Launch.scala:18)
            at xsbt.boot.Boot$.runImpl(Boot.scala:41)
            at xsbt.boot.Boot$.main(Boot.scala:17)
            at xsbt.boot.Boot.main(Boot.scala)
    Caused by: java.io.IOException: Too many open files in system
            at java.io.UnixFileSystem.createFileExclusively(Native Method)
            at java.io.File.createNewFile(File.java:1012)
            at sbt.IO$$anonfun.apply$mcZ$sp(IO.scala:196)
            at sbt.IO$$anonfun.apply(IO.scala:196)
            at sbt.IO$$anonfun.apply(IO.scala:196)
            at sbt.ErrorHandling$.translate(ErrorHandling.scala:10)
            at sbt.IO$.touch(IO.scala:196)
            at sbt.std.Streams$$anon$$anon.make(Streams.scala:129)
            at sbt.std.Streams$$anon$$anon.binary(Streams.scala:116)
            at sbt.SessionVar$$anonfun$persist.apply(SessionVar.scala:27)
            at sbt.SessionVar$$anonfun$persist.apply(SessionVar.scala:26)
            at sbt.std.Streams$class.use(Streams.scala:75)
            at sbt.std.Streams$$anon.use(Streams.scala:100)
            at sbt.SessionVar$.persist(SessionVar.scala:26)
            at sbt.SessionVar$.persistAndSet(SessionVar.scala:21)
            at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$$anonfun$apply.apply(Project.scala:563)
            at sbt.Project$RichTaskSessionVar$$anonfun$storeAs$$anonfun$apply.apply(Project.scala:563)
            at sbt.SessionVar$$anonfun$$anonfun$apply.apply(SessionVar.scala:40)
            at sbt.SessionVar$$anonfun$$anonfun$apply.apply(SessionVar.scala:40)
            at scala.Function$$anonfun$chain$$anonfun$apply.apply(Function.scala:24)
            at scala.Function$$anonfun$chain$$anonfun$apply.apply(Function.scala:24)
            at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:51)
            at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:60)
            at scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:47)
            at scala.collection.TraversableOnce$class.$div$colon(TraversableOnce.scala:138)
            at scala.collection.AbstractTraversable.$div$colon(Traversable.scala:105)
            at scala.Function$$anonfun$chain.apply(Function.scala:24)
            at sbt.EvaluateTask$.applyResults(EvaluateTask.scala:387)
            at sbt.EvaluateTask$.liftedTree1(EvaluateTask.scala:361)
            at sbt.EvaluateTask$.run(EvaluateTask.scala:358)
            at sbt.EvaluateTask$.runTask(EvaluateTask.scala:378)
            at sbt.Aggregation$$anonfun.apply(Aggregation.scala:64)
            at sbt.Aggregation$$anonfun.apply(Aggregation.scala:62)
            at sbt.EvaluateTask$.withStreams(EvaluateTask.scala:314)
            at sbt.Aggregation$.timedRun(Aggregation.scala:62)
            at sbt.Aggregation$.runTasks(Aggregation.scala:71)
            at sbt.Aggregation$$anonfun$applyTasks.apply(Aggregation.scala:32)
            at sbt.Aggregation$$anonfun$applyTasks.apply(Aggregation.scala:31)
            at sbt.Command$$anonfun$applyEffect$$anonfun$apply.apply(Command.scala:61)
            at sbt.Command$$anonfun$applyEffect$$anonfun$apply.apply(Command.scala:61)
            at sbt.Aggregation$$anonfun$evaluatingParser$$anonfun$apply.apply(Aggregation.scala:153)
            at sbt.Aggregation$$anonfun$evaluatingParser$$anonfun$apply.apply(Aggregation.scala:152)
            at sbt.Act$$anonfun$sbt$Act$$actParser0$$anonfun$sbt$Act$$anonfun$$evaluate$$anonfun$apply.apply(Act.scala:244)
            at sbt.Act$$anonfun$sbt$Act$$actParser0$$anonfun$sbt$Act$$anonfun$$evaluate$$anonfun$apply.apply(Act.scala:241)
            at sbt.Command$.process(Command.scala:93)
            at sbt.MainLoop$$anonfun$$anonfun$apply.apply(MainLoop.scala:96)
            at sbt.MainLoop$$anonfun$$anonfun$apply.apply(MainLoop.scala:96)
            at sbt.State$$anon.process(State.scala:184)
            at sbt.MainLoop$$anonfun.apply(MainLoop.scala:96)
            at sbt.MainLoop$$anonfun.apply(MainLoop.scala:96)
            at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
            at sbt.MainLoop$.next(MainLoop.scala:96)
            at sbt.MainLoop$.run(MainLoop.scala:89)
            at sbt.MainLoop$$anonfun$runWithNewLog.apply(MainLoop.scala:68)
            at sbt.MainLoop$$anonfun$runWithNewLog.apply(MainLoop.scala:63)
            at sbt.Using.apply(Using.scala:24)
            at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
            at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
            at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
            at sbt.MainLoop$.runLogged(MainLoop.scala:22)
            at sbt.StandardMain$.runManaged(Main.scala:54)
            at sbt.xMain.run(Main.scala:29)
            at xsbt.boot.Launch$$anonfun$run.apply(Launch.scala:109)
            at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
            at xsbt.boot.Launch$.run(Launch.scala:109)
            at xsbt.boot.Launch$$anonfun$apply.apply(Launch.scala:35)
            at xsbt.boot.Launch$.launch(Launch.scala:117)
            at xsbt.boot.Launch$.apply(Launch.scala:18)
            at xsbt.boot.Boot$.runImpl(Boot.scala:41)
            at xsbt.boot.Boot$.main(Boot.scala:17)
            at xsbt.boot.Boot.main(Boot.scala)
    [error] Could not create file [...omitted...]/target/streams/compile/$global/$global/discoveredMainClasses/datajava.io.IOException: Too man
    y open files in system
    [error] Use 'last' for the full log.

将 -XX:-MaxFDLimit 选项传递给 JVM 没有任何区别。

sbt的版本是0.13.9，sbt-native-packager是1.1.1（最新，也是用的v1.0.6）。

所以这是一个错误，还是我只需要在那个目录中放置很多文件？

Answer 1

据我所知，您的底层操作系统存在一些问题：

Caused by: java.io.IOException: Too many open files in system
        at java.io.UnixFileSystem.createFileExclusively(Native Method)

尝试在 Linux 中调整最大允许打开文件设置。例如 here 是一些命令：

If you are getting error “Too many open files (24)” then your application/command/script is hitting max open file limit allowed by linux. You need to increase open file limit as below:
Increase limit Per-User Limit
Open file: /etc/security/limits.conf

Paste following towards end:
*

hard    nofile      500000
*         soft    nofile      500000 root      hard    nofile      500000 root      soft    nofile      500000
500000 is fair number. I am not sure what is max limit but 999999 (Six-9) worked for me once as far as I remember.

Once you save file, you may need to logout and login again.
pam-limits
I read at many places that an extra step is neede for limit to change for daemon processes. I did not need following yet, but if above changes are not working for you, you may give this a try.

Open /etc/pam.d/common-session

Add following line:
session required
pam_limits.so
System-Wide Limit
Set this higher than user-limit set above.

Open /etc/sysctl.conf

Add following:
fs.file-max =
2097152
Run:
sysctl -p
Above will increase “total” number of files that can remain open system-wide.
Verify New Limits

Use following command to see max limit of file descriptors:
cat /proc/sys/fs/file-max
Hard Limit
ulimit
-Hn
Soft Limit
ulimit -Sn
if you are logged in as root:
Check limit for other user
Just replace www-data by linux username you wish to check limits for:
su - www-data -c
'ulimit -aHS' -s '/bin/bash'
Check limits of a running process:
Find process-id (PID):
ps aux |
grep process-name
Suppose, XXX is PID, then run following commands to check limits:
cat /proc/XXX/limits

映射目录时，sbt-native-packager 失败并显示 "Too many open files in system"

sbt-native-packager fails with "Too many open files in system" when mapping a directory

java

jvm

scala

sbt

sbt-native-packager