在 Athena SQL 查询中连接来自 Python Lambda 函数的变量

Question

我有一个 Python Lambda 函数，它在 Athena 中创建了一个 SQL table。如何在查询中正确连接变量？当我设置 LOCATION 值时，我收到以下错误响应。如果我对 LOCATION 值进行硬编码，该函数将成功运行。

LOCATION “”” + s3_bucket_test + “””

错误响应：

Response
{
  "errorMessage": "An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: line 1:8: mismatched input 'EXTERNAL'. Expecting: 'OR', 'SCHEMA', 'TABLE', 'VIEW'",
  "errorType": "InvalidRequestException",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 34, in lambda_handler\n    queryStart = client.start_query_execution(\n",
    "  File \"/var/runtime/botocore/client.py\", line 386, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 705, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

Lambda 函数：

import boto3
import json
import time

database = ‘daily_reports’
s3_bucket = 's3://test/’
s3_bucket_results = 's3://test/results’

query = ("""
          CREATE EXTERNAL TABLE IF NOT EXISTS `reports` (
              `timestamp` bigint,
              `user_id` string,
              `name` string
            )
            ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
            WITH SERDEPROPERTIES (
              'serialization.format' = '1'
            ) LOCATION “”” + s3_bucket + “””
            TBLPROPERTIES ('has_encrypted_data'='false');
        """)        


def lambda_handler(event, context):
    client = boto3.client('athena')
    queryStart = client.start_query_execution(
        QueryString = query,
        QueryExecutionContext = {
            'Database': database
        },
        ResultConfiguration = {
            'OutputLocation': s3_bucket_results
        }
    )

谢谢。

Answer 1

你试过Python的格式化方法吗？像这样

query = ("""
          CREATE EXTERNAL TABLE IF NOT EXISTS `reports` (
              `timestamp` bigint,
              `user_id` string,
              `name` string
            )
            ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
            WITH SERDEPROPERTIES (
              'serialization.format' = '1'
            ) LOCATION '{}' 
            TBLPROPERTIES ('has_encrypted_data'='false');
        """).format(s3_bucket)

在 Athena SQL 查询中连接来自 Python Lambda 函数的变量

Concatenate variable in Athena SQL query from Python Lambda function

sql

python-3.x

aws-lambda

amazon-athena