在 bazel 构建系统中配置相互依赖的项目(例如 tensorflow)的正确方法,以便 proto 导入按原样工作?
Correct way to configure interdependent projects (e.g. tensorflow) in bazel build system so proto imports work as is?
如标题所示,我 运行 遇到了一个问题,即 proto import 语句似乎与正确的路径不相关。具体来说,考虑目录中的目录结构(我们称之为 ~/base):
`>> tree -L 1
├── models
├── my-lib
| ├── nlp
| ├── BUILD
| └── nlp_parser.cc
| └── WORKSPACE
├── serving
└── tensorflow
对于那些不熟悉的人,models
(如 https://github.com/tensorflow/models/) has tensorflow (https://github.com/tensorflow/tensorflow)作为一个 git 子模块,serving
也是如此。由于这一点加上 tensorflow 的 git 子模块在不同的提交上有时不兼容,我从项目中删除了 git 子模块并将它们符号链接到最顶层目录的 tensorflow repo这样我就可以只管理一个张量流回购而不是 3 个。也就是说,我已经完成了以下操作:
`cd models/syntaxnet; rm -rf tensorflow; ln -s ../../tensorflow/ .; cd -`
`cd serving; rm -rf tensorflow tf_models; ln -s ../tensorflow/ .; ln -s ../models .`
现在我想在 my-lib
内构建一个依赖于 serving
、tensorflow
和 models
的目标。我将这些作为本地存储库添加到我的 WORKSPACE 中,如下所示 (cat my-lib/WORKSPACE
):
workspace(name = "myworkspace")
local_repository(
name = "org_tensorflow",
path = __workspace_dir__ + "/../tensorflow",
)
local_repository(
name = "syntaxnet",
path = __workspace_dir__ + "/../models/syntaxnet",
)
local_repository(
name = "tf_serving",
path = __workspace_dir__ + "/../serving",
)
load('@org_tensorflow//tensorflow:workspace.bzl', 'tf_workspace')
tf_workspace("~/base/tensorflow", "@org_tensorflow")
# ===== gRPC dependencies =====
bind(
name = "libssl",
actual = "@boringssl_git//:ssl",
)
bind(
name = "zlib",
actual = "@zlib_archive//:zlib",
)
这是我的构建文件 (cat my-lib/nlp/BUILD
):
load("@tf_serving//tensorflow_serving:serving.bzl", "serving_proto_library")
cc_binary(
name = "nlp_parser",
srcs = [ "nlp_parser.cc" ],
linkopts = ["-lm"],
deps = [
"@org_tensorflow//tensorflow/core:core_cpu",
"@org_tensorflow//tensorflow/core:framework",
"@org_tensorflow//tensorflow/core:lib",
"@org_tensorflow//tensorflow/core:protos_all_cc",
"@org_tensorflow//tensorflow/core:tensorflow",
"@syntaxnet//syntaxnet:parser_ops_cc",
"@syntaxnet//syntaxnet:sentence_proto",
"@tf_serving//tensorflow_serving/servables/tensorflow:session_bundle_config_proto",
"@tf_serving//tensorflow_serving/servables/tensorflow:session_bundle_factory",
"@org_tensorflow//tensorflow/contrib/session_bundle",
"@org_tensorflow//tensorflow/contrib/session_bundle:signature",
],
)
最后,这是构建的输出 (cd my-lib; bazel build nlp/nlp_parser --verbose_failures
):
INFO: Found 1 target...
ERROR: /home/blah/blah/external/org_tensorflow/tensorflow/core/debug/BUILD:33:1: null failed: linux-sandbox failed: error executing command
(cd /home/blah/blah/execroot/my-lib && \
exec env - \
/home/blah/blah/execroot/my-lib/_bin/linux-sandbox @/home/blah/blah/execroot/my-lib/bazel-sandbox/c65fa6b6-9b7d-4710-b19c-4d42a3e6a667-31.params -- bazel-out/host/bin/external/protobuf/protoc '--cpp_out=bazel-out/local-fastbuild/genfiles/external/org_tensorflow' '--plugin=protoc-gen-grpc=bazel-out/host/bin/external/grpc/grpc_cpp_plugin' '--grpc_out=bazel-out/local-fastbuild/genfiles/external/org_tensorflow' -Iexternal/org_tensorflow -Ibazel-out/local-fastbuild/genfiles/external/org_tensorflow -Iexternal/protobuf/src -Ibazel-out/local-fastbuild/genfiles/external/protobuf/src external/org_tensorflow/tensorflow/core/debug/debug_service.proto).
bazel-out/local-fastbuild/genfiles/external/protobuf/src: warning: directory does not exist.
tensorflow/core/util/event.proto: File not found.
tensorflow/core/debug/debug_service.proto: Import "tensorflow/core/util/event.proto" was not found or had errors.
tensorflow/core/debug/debug_service.proto:38:25: "Event" is not defined.
Target //nlp:nlp_parser failed to build
INFO: Elapsed time: 0.776s, Critical Path: 0.42s
在 WORKSPACE 中将模块添加为 local_repository 以便 proto 导入工作的正确方法是什么?
在 OS X 上构建依赖于 tensorflow 的项目后,我尝试构建我的项目后遇到了类似的问题。最终对我有用的是禁用沙盒--spawn_strategy=standalone
如标题所示,我 运行 遇到了一个问题,即 proto import 语句似乎与正确的路径不相关。具体来说,考虑目录中的目录结构(我们称之为 ~/base):
`>> tree -L 1
├── models
├── my-lib
| ├── nlp
| ├── BUILD
| └── nlp_parser.cc
| └── WORKSPACE
├── serving
└── tensorflow
对于那些不熟悉的人,models
(如 https://github.com/tensorflow/models/) has tensorflow (https://github.com/tensorflow/tensorflow)作为一个 git 子模块,serving
也是如此。由于这一点加上 tensorflow 的 git 子模块在不同的提交上有时不兼容,我从项目中删除了 git 子模块并将它们符号链接到最顶层目录的 tensorflow repo这样我就可以只管理一个张量流回购而不是 3 个。也就是说,我已经完成了以下操作:
`cd models/syntaxnet; rm -rf tensorflow; ln -s ../../tensorflow/ .; cd -`
`cd serving; rm -rf tensorflow tf_models; ln -s ../tensorflow/ .; ln -s ../models .`
现在我想在 my-lib
内构建一个依赖于 serving
、tensorflow
和 models
的目标。我将这些作为本地存储库添加到我的 WORKSPACE 中,如下所示 (cat my-lib/WORKSPACE
):
workspace(name = "myworkspace")
local_repository(
name = "org_tensorflow",
path = __workspace_dir__ + "/../tensorflow",
)
local_repository(
name = "syntaxnet",
path = __workspace_dir__ + "/../models/syntaxnet",
)
local_repository(
name = "tf_serving",
path = __workspace_dir__ + "/../serving",
)
load('@org_tensorflow//tensorflow:workspace.bzl', 'tf_workspace')
tf_workspace("~/base/tensorflow", "@org_tensorflow")
# ===== gRPC dependencies =====
bind(
name = "libssl",
actual = "@boringssl_git//:ssl",
)
bind(
name = "zlib",
actual = "@zlib_archive//:zlib",
)
这是我的构建文件 (cat my-lib/nlp/BUILD
):
load("@tf_serving//tensorflow_serving:serving.bzl", "serving_proto_library")
cc_binary(
name = "nlp_parser",
srcs = [ "nlp_parser.cc" ],
linkopts = ["-lm"],
deps = [
"@org_tensorflow//tensorflow/core:core_cpu",
"@org_tensorflow//tensorflow/core:framework",
"@org_tensorflow//tensorflow/core:lib",
"@org_tensorflow//tensorflow/core:protos_all_cc",
"@org_tensorflow//tensorflow/core:tensorflow",
"@syntaxnet//syntaxnet:parser_ops_cc",
"@syntaxnet//syntaxnet:sentence_proto",
"@tf_serving//tensorflow_serving/servables/tensorflow:session_bundle_config_proto",
"@tf_serving//tensorflow_serving/servables/tensorflow:session_bundle_factory",
"@org_tensorflow//tensorflow/contrib/session_bundle",
"@org_tensorflow//tensorflow/contrib/session_bundle:signature",
],
)
最后,这是构建的输出 (cd my-lib; bazel build nlp/nlp_parser --verbose_failures
):
INFO: Found 1 target...
ERROR: /home/blah/blah/external/org_tensorflow/tensorflow/core/debug/BUILD:33:1: null failed: linux-sandbox failed: error executing command
(cd /home/blah/blah/execroot/my-lib && \
exec env - \
/home/blah/blah/execroot/my-lib/_bin/linux-sandbox @/home/blah/blah/execroot/my-lib/bazel-sandbox/c65fa6b6-9b7d-4710-b19c-4d42a3e6a667-31.params -- bazel-out/host/bin/external/protobuf/protoc '--cpp_out=bazel-out/local-fastbuild/genfiles/external/org_tensorflow' '--plugin=protoc-gen-grpc=bazel-out/host/bin/external/grpc/grpc_cpp_plugin' '--grpc_out=bazel-out/local-fastbuild/genfiles/external/org_tensorflow' -Iexternal/org_tensorflow -Ibazel-out/local-fastbuild/genfiles/external/org_tensorflow -Iexternal/protobuf/src -Ibazel-out/local-fastbuild/genfiles/external/protobuf/src external/org_tensorflow/tensorflow/core/debug/debug_service.proto).
bazel-out/local-fastbuild/genfiles/external/protobuf/src: warning: directory does not exist.
tensorflow/core/util/event.proto: File not found.
tensorflow/core/debug/debug_service.proto: Import "tensorflow/core/util/event.proto" was not found or had errors.
tensorflow/core/debug/debug_service.proto:38:25: "Event" is not defined.
Target //nlp:nlp_parser failed to build
INFO: Elapsed time: 0.776s, Critical Path: 0.42s
在 WORKSPACE 中将模块添加为 local_repository 以便 proto 导入工作的正确方法是什么?
在 OS X 上构建依赖于 tensorflow 的项目后,我尝试构建我的项目后遇到了类似的问题。最终对我有用的是禁用沙盒--spawn_strategy=standalone