创建和替换 BigQuery 表

Question

如何创建和替换现有的 BigQuery table？我使用数据实验室定义 BigQuery 查询并将结果写入 BigQuery tables.

我发现最有效的方法是：

%%bq query --name helloWorld
Select * from someTable

其次是

%%bq execute --table schemaName.destination_table --query helloWorld

但是我每次都必须手动删除 table

从命令行我可以执行类似的东西：

bq query --destination_table [PROJECT_ID]:[DATASET].[TABLE] --replace '[QUERY]'

python/Datalab中是否有类似的功能？

Answer 1

使用 --mode 或 -m 参数。例如：

%%bq execute --table schemaName.destination_table --query helloWorld --mode overwrite

查看更多选项运行 %%bq execute --help:

usage: %bq execute [-h] [-nc] [-b BILLING] [-m {create,append,overwrite}] [-l]
                   -q QUERY [-t TABLE] [--to-dataframe]
                   [--dataframe-start-row DATAFRAME_START_ROW]
                   [--dataframe-max-rows DATAFRAME_MAX_ROWS] [-v]

Execute a BigQuery SQL query and optionally send the results to a named table.
The cell can optionally contain arguments for expanding variables in the
query.

optional arguments:
  -h, --help            show this help message and exit
  -nc, --nocache        Don't use previously cached results
  -b BILLING, --billing BILLING
                        BigQuery billing tier
  -m {create,append,overwrite}, --mode {create,append,overwrite}
                        The table creation mode
  -l, --large           Whether to allow large results
  -q QUERY, --query QUERY
                        The name of query to run
  -t TABLE, --table TABLE
                        Target table name
  --to-dataframe        Convert the result into a dataframe
  --dataframe-start-row DATAFRAME_START_ROW
                        Row of the table to start the dataframe export
  --dataframe-max-rows DATAFRAME_MAX_ROWS
                        Upper limit on number of rows to export to the
                        dataframe
  -v, --verbose         Show the expanded SQL that is being executed

创建和替换 BigQuery 表

Create and replace BigQuery tables

python

google-bigquery

google-cloud-datalab