Azure DataWarehouse 使用外部 Table 加载 CSV
Azure DataWarehouse load CSV with external Table
我找不到一个完整的示例来说明如何将带有外部 table 的 CSV 文件直接加载到 Sql 数据仓库中。
该文件位于存储帐户 https://tstodummy.blob.core.windows.net/
Blob 容器 referencedata-in,文件夹 csv-uploads,文件 something.csv。
这是我的代码
CREATE DATABASE SCOPED CREDENTIAL tstodummy_refdata_credential
WITH IDENTITY = 'USER',
SECRET = '....'
GO
CREATE EXTERNAL DATA SOURCE tstodummy_referencedata
WITH ( TYPE = HADOOP,
LOCATION = 'wasb://referencedata-in@tstodummy.blob.core.windows.net',
CREDENTIAL = tstodummy_refdata_credential);
GO
CREATE EXTERNAL FILE FORMAT aps_bma_referencedata_ff
WITH (FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS(
FIELD_TERMINATOR = ';',
STRING_DELIMITER = '"',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = True)
)
CREATE EXTERNAL TABLE [stg_aps_bma_refdata].[PlanDeMaintenance]
( [Version] VARCHAR(255) NULL
, [Description] VARCHAR(255) NULL
, [Date_Start] VARCHAR(255) NULL
, [Date_Stop] VARCHAR(255) NULL
) WITH ( LOCATION = '\referencedata-in\csv-uploads\PlanDeMaintanance'
, DATA_SOURCE = tstodummy_referencedata
, FILE_FORMAT = aps_bma_referencedata_ff
, REJECT_TYPE = VALUE
, REJECT_VALUE = 0
)
我一直在玩 Location 中的各种组合...但是,Nogo
错误是
Msg 105002, Level 16, State 1, Line 26
EXTERNAL TABLE access failed because the specified path name '/referencedata-in/csv-uploads/PlanDeMaintanance.csv' does not exist. Enter a valid path and try again.
我看不到你的存储结构,但我想你会发现问题出在外部 table 位置包含“/referencedata-in”。
一件小事,您可能还想考虑在存储 URL 上添加一个 "wasbs" 前缀,以便将 SSL 加密应用于传输。
最后,这在其他人遇到销售问题时起到了作用。
我还没有在这段代码中添加我收到的评论,同时完成。
CREATE EXTERNAL DATA SOURCE tsto_referencedata
WITH ( TYPE = HADOOP,
LOCATION = 'wasb://referencedata-in@tsto.blob.core.windows.net',
CREDENTIAL = tsto_refdata_credential);
GO
CREATE EXTERNAL FILE FORMAT aps_bma_referencedata_ff
WITH (FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS(
FIELD_TERMINATOR = ',',
STRING_DELIMITER = '"',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = True)
)
CREATE EXTERNAL TABLE [stg_aps_bma_refdata].[PlanDeMaintenance.csv]
( [Version] VARCHAR(255) NULL
, [Description] VARCHAR(255) NULL
, [Date_Start] VARCHAR(255) NULL
, [Date_Stop] VARCHAR(255) NULL
) WITH ( LOCATION = '/csv-uploads/PlanDeMaintenance.csv'
, DATA_SOURCE = tsto_referencedata
, FILE_FORMAT = aps_bma_referencedata_ff
, REJECT_TYPE = VALUE
, REJECT_VALUE = 0
)
我找不到一个完整的示例来说明如何将带有外部 table 的 CSV 文件直接加载到 Sql 数据仓库中。 该文件位于存储帐户 https://tstodummy.blob.core.windows.net/ Blob 容器 referencedata-in,文件夹 csv-uploads,文件 something.csv。 这是我的代码
CREATE DATABASE SCOPED CREDENTIAL tstodummy_refdata_credential
WITH IDENTITY = 'USER',
SECRET = '....'
GO
CREATE EXTERNAL DATA SOURCE tstodummy_referencedata
WITH ( TYPE = HADOOP,
LOCATION = 'wasb://referencedata-in@tstodummy.blob.core.windows.net',
CREDENTIAL = tstodummy_refdata_credential);
GO
CREATE EXTERNAL FILE FORMAT aps_bma_referencedata_ff
WITH (FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS(
FIELD_TERMINATOR = ';',
STRING_DELIMITER = '"',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = True)
)
CREATE EXTERNAL TABLE [stg_aps_bma_refdata].[PlanDeMaintenance]
( [Version] VARCHAR(255) NULL
, [Description] VARCHAR(255) NULL
, [Date_Start] VARCHAR(255) NULL
, [Date_Stop] VARCHAR(255) NULL
) WITH ( LOCATION = '\referencedata-in\csv-uploads\PlanDeMaintanance'
, DATA_SOURCE = tstodummy_referencedata
, FILE_FORMAT = aps_bma_referencedata_ff
, REJECT_TYPE = VALUE
, REJECT_VALUE = 0
)
我一直在玩 Location 中的各种组合...但是,Nogo 错误是
Msg 105002, Level 16, State 1, Line 26
EXTERNAL TABLE access failed because the specified path name '/referencedata-in/csv-uploads/PlanDeMaintanance.csv' does not exist. Enter a valid path and try again.
我看不到你的存储结构,但我想你会发现问题出在外部 table 位置包含“/referencedata-in”。
一件小事,您可能还想考虑在存储 URL 上添加一个 "wasbs" 前缀,以便将 SSL 加密应用于传输。
最后,这在其他人遇到销售问题时起到了作用。 我还没有在这段代码中添加我收到的评论,同时完成。
CREATE EXTERNAL DATA SOURCE tsto_referencedata
WITH ( TYPE = HADOOP,
LOCATION = 'wasb://referencedata-in@tsto.blob.core.windows.net',
CREDENTIAL = tsto_refdata_credential);
GO
CREATE EXTERNAL FILE FORMAT aps_bma_referencedata_ff
WITH (FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS(
FIELD_TERMINATOR = ',',
STRING_DELIMITER = '"',
FIRST_ROW = 2,
USE_TYPE_DEFAULT = True)
)
CREATE EXTERNAL TABLE [stg_aps_bma_refdata].[PlanDeMaintenance.csv]
( [Version] VARCHAR(255) NULL
, [Description] VARCHAR(255) NULL
, [Date_Start] VARCHAR(255) NULL
, [Date_Stop] VARCHAR(255) NULL
) WITH ( LOCATION = '/csv-uploads/PlanDeMaintenance.csv'
, DATA_SOURCE = tsto_referencedata
, FILE_FORMAT = aps_bma_referencedata_ff
, REJECT_TYPE = VALUE
, REJECT_VALUE = 0
)