使用脚本调度查询

Question

请问是否可以使用脚本进行朗姆酒查询调度？

至于创建table，我们可以使用脚本

CREATE TABLE dataset.xxx AS

...

除了创建调度程序而不是单击 'Schedule Query' 按钮之外，还有什么方法可以做到这一点吗？

Answer 1

根据 documentation，要安排查询，您可以使用以下方法之一：

BigQuery 控制台并点击您在问题中提到的“计划查询”。
bq 命令
Python API

我将与您分享相同的示例，首先使用 bq 命令。在云 Shell 环境中，您可以执行以下命令：

bq query \
    --use_legacy_sql=false \
    --destination_table=mydataset.mytable \
    --display_name='My Scheduled Query' \
    --replace=true \
    'SELECT
      1
FROM
     mydataset.test'

此外，使用 bq 命令 您还可以使用其他标志，描述 here.

其次，使用 Python API，您可以使用 DataTransferServiceClient 配置您的计划查询，它允许您通过 json 传递所有查询配置字典，例如 documentation 及以下的示例：

from google.cloud import bigquery_datatransfer_v1
import google.protobuf.json_format

client = bigquery_datatransfer_v1.DataTransferServiceClient()

# TODO(developer): Set the project_id to the project that contains the
#                  destination dataset.
# project_id = "your-project-id"

# TODO(developer): Set the destination dataset. The authorized user must
#                  have owner permissions on the dataset.
# dataset_id = "your_dataset_id"

# TODO(developer): The first time you run this sample, set the
# authorization code to a value from the URL:
# https://www.gstatic.com/bigquerydatatransfer/oauthz/auth?client_id=433065040935-hav5fqnc9p9cht3rqneus9115ias2kn1.apps.googleusercontent.com&scope=https://www.googleapis.com/auth/bigquery%20https://www.googleapis.com/auth/drive&redirect_uri=urn:ietf:wg:oauth:2.0:oob
#
# authorization_code = "_4/ABCD-EFGHIJKLMNOP-QRSTUVWXYZ"
#
# You can use an empty string for authorization_code in subsequent runs of
# this code sample with the same credentials.
#
# authorization_code = ""

# Use standard SQL syntax for the query.
query_string = """
SELECT
  CURRENT_TIMESTAMP() as current_time,
  @run_time as intended_run_time,
  @run_date as intended_run_date,
  17 as some_integer
"""

parent = client.project_path(project_id)

transfer_config = google.protobuf.json_format.ParseDict(
    {
        "destination_dataset_id": dataset_id,
        "display_name": "Your Scheduled Query Name",
        "data_source_id": "scheduled_query",
        "params": {
            "query": query_string,
            "destination_table_name_template": "your_table_{run_date}",
            "write_disposition": "WRITE_TRUNCATE",
            "partitioning_field": "",
        },
        "schedule": "every 24 hours",
    },
    bigquery_datatransfer_v1.types.TransferConfig(),
)

response = client.create_transfer_config(
    parent, transfer_config, authorization_code=authorization_code
)

print("Created scheduled query '{}'".format(response.name))

使用脚本调度查询

Scheduling Query by using Script

scheduler

google-bigquery

taskscheduler