外部 table Azure Synapse 不返回数据

Question

我正在尝试创建 EXTERNAL TABLE, mapping it from a Blob Storage following this tutorial Load Contoso retail data to Synapse SQL。

但是当我查询 table 时出现此错误： Failed to execute query. Error: HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Too many columns in the line.

我的文件配置是：

A: Create a database scoped credential

CREATE DATABASE SCOPED CREDENTIAL ServiceNow_AzureStorageCredential_ADSL
WITH
     IDENTITY = 'usr_adsl_servicnow'
    ,SECRET = 'my_key'
;

B: Create an external data source

CREATE EXTERNAL DATA SOURCE ServiceNowBlobStorage
WITH (
    TYPE = HADOOP,
    LOCATION = 'abfss://<servicenow container>@<account storage>.blob.core.windows.net',
    CREDENTIAL = ServiceNow_AzureStorageCredential_ADSL
);

C: Create the file format to be read from blob storage

CREATE EXTERNAL FILE FORMAT ServiceNowFileFormatCSV
WITH
(    FORMAT_TYPE = DELIMITEDTEXT
,    FORMAT_OPTIONS    (   FIELD_TERMINATOR = ','
                      ,    STRING_DELIMITER = '"'
                      ,    FIRST_ROW = 2
                      ,    DATE_FORMAT = 'dd/MM/yyyy HH:mm:ss'
                      ,    USE_TYPE_DEFAULT = TRUE
                      ,    Encoding = 'UTF8'
                    )
);

D: Create the external table

CREATE EXTERNAL TABLE [asb].[incidents] (
    [number] [nvarchar](30) NOT NULL,
    [opened] [datetime] NOT NULL,
    [resolved] [datetime] NULL,
    [updated] [datetime] NULL,
    [short_description] [nvarchar](2000) NOT NULL,
    [urgency] [nvarchar](65) NOT NULL,
    [resolve_time] [nvarchar](100) NULL,
    [business_service] [nvarchar](100) NULL,
    [what_is_the_system] [nvarchar](100) NULL,
    [problem] [nvarchar](65) NULL,
    [parent] [nvarchar](65) NULL,
    [where_is_the_problem] [nvarchar](100) NULL,
    [child_incidents] [nvarchar](30) NULL,
    [parent_incident] [nvarchar](65) NULL,
    [impact] [nvarchar](65) NULL,
    [severity] [nvarchar](65) NULL,
    [incident_state] [nvarchar](100) NULL,
    [company] [nvarchar](30) NULL,
    [business_duration] [nvarchar](65) NULL,
    [duration] [nvarchar](65) NULL,
    [created] [nvarchar](65) NULL,
    [catalog_item] [nvarchar](100) NULL,
    [priority] [nvarchar](100) NULL,
    [state] [nvarchar](65) NULL,
    [category] [nvarchar](30) NULL,
    [assignment_group] [nvarchar](100) NULL,
    [location] [nvarchar](200) NULL,
    [ETLLoadID] [nvarchar](65) NULL,
    [LoadDate] [nvarchar](65) NULL,
    [UpdateDate] [nvarchar](65) NULL
)
WITH
(
        LOCATION='servicenow-tables/incidents/incidents.csv'
    ,   DATA_SOURCE = ServiceNowBlobStorage
    ,   FILE_FORMAT = ServiceNowFileFormatCSV
    ,   REJECT_TYPE = VALUE
    ,   REJECT_VALUE = 0
);

本次测试我只使用nvarchar数据类型以避免转换错误Error converting data type VARCHAR to DATETIME。

我的文件格式是：

number,opened,resolved,updated,short_description,urgency,task_type,resolve_time,business_service,what_is_the_system,problem,parent,where_is_the_problem,child_incidents,parent_incident,impact,severity,incident_state,company,business_duration,duration,created,catalog_item,priority,state,category,assignment_group,location
"INC0020620","15/05/2020 10:42:39","19/05/2020 12:49:36","26/05/2020 13:00:02","Problemas de divergência nos valores","3 - Baixo(a)","Incidente","45,2620486","PDV",,"PRB0040714","",,"0","","1 - Alto(a)","3 - Baixo(a)","Encerrado","Lojas  S.A.","18 Horas 6 Minutos","4 Dias 2 Horas 6 Minutos","15/05/2020 10:42:39","Problemas de divergência nos valores","3 - Moderado","Encerrado","Sistemas/Aplicações","TI_N2_SIS","ADMINISTRACAO"

我尝试了很多方法来解决这个问题，但都没有成功。

Answer 1

我只是简单地搜索了顶行的“,”，它给出了 27 个计数，这意味着我们有 28 个字段。在底线上做同样的事情会得到 28 个计数，这意味着我们有 29 个字段。所以你在第二行有更多的额外价值因此错误，我认为问题低于价值

"PRB0040714","","0

希望这有帮助

外部 table Azure Synapse 不返回数据

External table Azure Synapse does't returning data

azure

azure-sql-data-warehouse

azure-synapse