Gradle 使用 Chaquopy 构建内存问题
Gradle build memory issues with Chaquopy
我有一个 Android 相机应用程序和一个单独的 Python 图像处理管道，我想使用 Chaquopy 将其与该应用程序集成。然而，python 代码本身很大，依赖于大约 50 个模块，例如 tensorflow、opencv、scikit-image、numpy 和 scipy 等。我可以对来自相机应用程序的传入帧执行 运行 玩具 python 功能。但是，我什至无法构建完整的管道，因为 Gradle 退出并出现以下错误：
任务：应用程序：生成调试PythonRequirementsAssets
任务：应用程序：generateDebugPythonBuildAssets 失败
内存不足：Java 堆 space。
请为项目的 gradle.properties 文件中的 Gradle 分配更多内存。
例如，以下行在 gradle.properties 文件中将最大 Java 堆大小设置为 1,024 MB：
它在“Application:generateDebugPythonRequirementsAssets”或
“Application:generateDebugPythonBuildAssets”，取决于 requirememnts.txt
中有多少个模块
我曾尝试按照建议增加堆 space，最多 128 GB，但没有帮助。
如何减少内存需求以适应一些合理的 space？
编辑
这是来自 build.gradle
的 pip 块
pip {
install "-r", "requirements.txt"
}
requirements.txt 文件具有以下内容：
absl-py==0.9.0
ansicolors==1.1.8
astor==0.8.1
cachetools==4.0.0
certifi==2019.11.28
chardet==3.0.4
cycler==0.10.0
decorator==4.4.2
gast==0.2.2
google-auth==1.11.3
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio
h5py==2.10.0
idna==2.9
imageio==2.8.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.2.1
matplotlib
networkx==2.4
nose==1.3.7
numpy
oauthlib==3.1.0
opencv-contrib-python==4.1.2.30
opt-einsum==3.2.0
Pillow==7.0.0
protobuf==3.11.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.6
python-dateutil==2.8.1
PyWavelets==1.1.1
requests==2.23.0
requests-oauthlib==1.3.0
rsa==4.0
scikit-image==0.16.2
scipy==1.4.1
six==1.14.0
tensorboard==2.1.1
tensorflow==2.1.0
termcolor==1.1.0
urllib3==1.25.8
Werkzeug==1.0.0
wrapt==1.12.1
编辑
这是完整的堆栈跟踪
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':Application:generateDebugPythonBuildAssets'.
> Java heap space
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':Application:generateDebugPythonBuildAssets'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.accept(ExecuteActionsTaskExecuter.java:151)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.accept(ExecuteActionsTaskExecuter.java:148)
at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:191)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:141)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:62)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:108)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.executeTask(EventFiringTaskExecuter.java:73)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.call(EventFiringTaskExecuter.java:52)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.call(EventFiringTaskExecuter.java:49)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:49)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:134)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:129)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl.run(ManagedExecutorImpl.java:46)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.chaquo.python.PythonPlugin.hashAssets(PythonPlugin.groovy:761)
at com.chaquo.python.PythonPlugin$hashAssets.callCurrent(Unknown Source)
at com.chaquo.python.PythonPlugin.hashAssets(PythonPlugin.groovy:749)
at com.chaquo.python.PythonPlugin$_createAssetsTasks_closure24$_closure61.doCall(PythonPlugin.groovy:679)
通过将以下内容添加到 gradle.properties 文件
解决了这个问题
org.gradle.jvmargs=-Xmx16G
org.gradle.workers.max=1
我有一个 Android 相机应用程序和一个单独的 Python 图像处理管道，我想使用 Chaquopy 将其与该应用程序集成。然而，python 代码本身很大，依赖于大约 50 个模块，例如 tensorflow、opencv、scikit-image、numpy 和 scipy 等。我可以对来自相机应用程序的传入帧执行 运行 玩具 python 功能。但是，我什至无法构建完整的管道，因为 Gradle 退出并出现以下错误：
任务：应用程序：生成调试PythonRequirementsAssets
任务：应用程序：generateDebugPythonBuildAssets 失败
内存不足：Java 堆 space。 请为项目的 gradle.properties 文件中的 Gradle 分配更多内存。 例如，以下行在 gradle.properties 文件中将最大 Java 堆大小设置为 1,024 MB：
它在“Application:generateDebugPythonRequirementsAssets”或 “Application:generateDebugPythonBuildAssets”，取决于 requirememnts.txt
中有多少个模块我曾尝试按照建议增加堆 space，最多 128 GB，但没有帮助。
如何减少内存需求以适应一些合理的 space？
编辑
这是来自 build.gradle
的 pip 块pip {
install "-r", "requirements.txt"
}
requirements.txt 文件具有以下内容：
absl-py==0.9.0
ansicolors==1.1.8
astor==0.8.1
cachetools==4.0.0
certifi==2019.11.28
chardet==3.0.4
cycler==0.10.0
decorator==4.4.2
gast==0.2.2
google-auth==1.11.3
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio
h5py==2.10.0
idna==2.9
imageio==2.8.0
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.2.1
matplotlib
networkx==2.4
nose==1.3.7
numpy
oauthlib==3.1.0
opencv-contrib-python==4.1.2.30
opt-einsum==3.2.0
Pillow==7.0.0
protobuf==3.11.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.6
python-dateutil==2.8.1
PyWavelets==1.1.1
requests==2.23.0
requests-oauthlib==1.3.0
rsa==4.0
scikit-image==0.16.2
scipy==1.4.1
six==1.14.0
tensorboard==2.1.1
tensorflow==2.1.0
termcolor==1.1.0
urllib3==1.25.8
Werkzeug==1.0.0
wrapt==1.12.1
编辑
这是完整的堆栈跟踪
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':Application:generateDebugPythonBuildAssets'.
> Java heap space
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':Application:generateDebugPythonBuildAssets'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.accept(ExecuteActionsTaskExecuter.java:151)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.accept(ExecuteActionsTaskExecuter.java:148)
at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:191)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:141)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:62)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:108)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.executeTask(EventFiringTaskExecuter.java:73)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.call(EventFiringTaskExecuter.java:52)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.call(EventFiringTaskExecuter.java:49)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:49)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:134)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:129)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl.run(ManagedExecutorImpl.java:46)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: java.lang.OutOfMemoryError: Java heap space
at com.chaquo.python.PythonPlugin.hashAssets(PythonPlugin.groovy:761)
at com.chaquo.python.PythonPlugin$hashAssets.callCurrent(Unknown Source)
at com.chaquo.python.PythonPlugin.hashAssets(PythonPlugin.groovy:749)
at com.chaquo.python.PythonPlugin$_createAssetsTasks_closure24$_closure61.doCall(PythonPlugin.groovy:679)
通过将以下内容添加到 gradle.properties 文件
解决了这个问题org.gradle.jvmargs=-Xmx16G
org.gradle.workers.max=1