如何 运行 来自 python Google API 私有子网上客户端库的数据流
How to run Dataflow from python Google API Client Libraries on private subnetwork
我正在尝试使用 python google api 客户端库启动 Dataflow 作业。之前一切正常,直到我们不得不从默认子网迁移到另一个私有子网。之前我使用以下代码启动数据流作业:
request = dataflow.projects().locations().templates().launch(
projectId = PROJECT_ID,
location = REGION,
gcsPath = TEMPLATE_LOCATION,
body = {
'jobName': job_name,
'parameters': job_parameters,
}
)
response = request.execute()
但是现在作业会失败,因为默认的子网已经不存在了,我现在需要指定使用data-subnet
子网。
来自 this documentation and also other question, the solution would be trivial if i were to launch the script from command line by adding the flag --subnetwork regions/$REGION/subnetworks/$PRIVATESUBNET
. However my case is different becuase I am trying to do it from code, and in the documentation 我找不到任何子网参数选项。
您可以像这样为管道指定自定义子网
request = dataflow.projects().locations().templates().launch(
projectId = PROJECT_ID,
location = REGION,
gcsPath = TEMPLATE_LOCATION,
body = {
'jobName': job_name,
'parameters': job_parameters,
'environment': {
'subnetwork': SUBNETWORK,
}
}
)
response = request.execute()
确保 SUBNETWORK 的格式为 "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/subnetworks/<subnetwork-name>"
我正在尝试使用 python google api 客户端库启动 Dataflow 作业。之前一切正常,直到我们不得不从默认子网迁移到另一个私有子网。之前我使用以下代码启动数据流作业:
request = dataflow.projects().locations().templates().launch(
projectId = PROJECT_ID,
location = REGION,
gcsPath = TEMPLATE_LOCATION,
body = {
'jobName': job_name,
'parameters': job_parameters,
}
)
response = request.execute()
但是现在作业会失败,因为默认的子网已经不存在了,我现在需要指定使用data-subnet
子网。
来自 this documentation and also --subnetwork regions/$REGION/subnetworks/$PRIVATESUBNET
. However my case is different becuase I am trying to do it from code, and in the documentation 我找不到任何子网参数选项。
您可以像这样为管道指定自定义子网
request = dataflow.projects().locations().templates().launch(
projectId = PROJECT_ID,
location = REGION,
gcsPath = TEMPLATE_LOCATION,
body = {
'jobName': job_name,
'parameters': job_parameters,
'environment': {
'subnetwork': SUBNETWORK,
}
}
)
response = request.execute()
确保 SUBNETWORK 的格式为 "https://www.googleapis.com/compute/v1/projects/<project-id>/regions/<region>/subnetworks/<subnetwork-name>"