由于 CSV 数据文件中的列值无效，批量加载失败

Question

我正在尝试将数据从 .csv 文件导入到 SQL server table。这适用于运行和 Microsoft SQL Server Management Studio (SSMS) 中的 SQL。但是，当我尝试使用 pyodbc 从 Python 执行此操作时，出现以下错误：

pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][SQL Server Native Client 11.0]
[SQL Server]Bulk load failed due to invalid column value in CSV data file
 C:/~pathToFile~/file.csv in row 2, column 38. (4879) (SQLExecDirectW);
 [42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]The OLE DB
 provider "BULK" for linked server "(null)" reported an error.
 The provider did not give any information about the error. (7399);
 [42000] [Microsoft][SQL Server Native Client 11.0][SQL Server]Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)". (7330)')

到目前为止我的代码是：

import pyodbc 

# ------------------------------------------------------
# DEFINE FUNCTIONS
# ------------------------------------------------------

# Define a database command
def DB(SQL):
    cnxn = pyodbc.connect("Driver={SQL Server Native Client 11.0};"
                          "Server=~myServer~;"
                          "Database=~myDB~;"
                          "Trusted_Connection=yes;")
    try:
        with cnxn.cursor() as cursor:
            cursor.execute(SQL)
        cnxn.commit()
    finally:
        cnxn.close()

# ------------------------------------------------------
# Import from .csv file to table
# ------------------------------------------------------

sql = '''BULK INSERT dbo.~myDB~
FROM 'C:/~pathToFile~/file.csv'
WITH ( 
    FORMAT='CSV', 
    FIRSTROW = 2, 
    ROWTERMINATOR = '\n', 
    FIELDQUOTE= '"',
    TABLOCK  
)'''
DB(sql)

这是我要导入的 .csv 文件的前几行：

SITE_ID,FIELD_SAMPLE_ID,LOCATION_ID,SAMPLE_DATE,PARAMETER_NAME,REPORT_RESULT,REPORT_UNITS,LAB_QUALIFIER,DETECTED,SAMPLE_MATRIX,SAMPLE_PURPOSE,SAMPLE_TYPE,SAMPLE_TIME,LATITUDE_(DECIMAL),LONGITUDE_(DECIMAL),FILTERED,FIELD_SAMPLE_COMMENTS,LAB_MATRIX,COC_#,LAB_METHOD,REPORT_DETECTION_LIMIT,SOURCE_FILENAME,WTR_SOURCE_FLOW,VALIDATION_QUALIFIER,VALIDATION_REASON_CODES,ANALYSIS_DATE,RESULT_TYPE,PARAMETER_CODE,LAB_RESULT,DILUTION_FACTOR,METHOD_DETECTION_LIMIT,INSTRUMENT_DETECTION_LIMIT,ANALYSIS_TYPE_CODE,ANALYSIS_TIME,QC_BATCH_SEQUENCE_#,SAMPLE_RESULT_COMMENTS,LAB_SAMPLE_ID,FIELD_SAMPLE_RESULT_RECORD_ID
"N3B","CAPA-08-11017","03-B-10","03-17-2008","RDX","0.325","ug/L","U","N","W","REG","WG","10:40","35.873716600000","-106.330115800000","N",,"W","08-824","SW-846:8321A_MOD","0.33",,"N","U","U_LAB","03-26-2008","TRG","121-82-4","0.325","2","0.13",,"INIT","00:00",,,"204935003","638"
"N3B","CAPA-08-13138","03-B-10","06-12-2008","RDX","0.325","ug/L","U","N","W","REG","WG","10:35","35.873716600000","-106.330115800000","N",,"W","08-1350","SW-846:8321A_MOD","0.33",,"N","U","U_LAB","06-24-2008","TRG","121-82-4","0.325","2","0.13",,"INIT","00:00",,,"210389014","638"
"N3B","CAPA-08-13139","03-B-10","06-12-2008","RDX","0.325","ug/L","U","N","W","FB","WG","10:35","35.873716600000","-106.330115800000","N",,"W","08-1350","SW-846:8321A_MOD","0.33",,"N","U","U_LAB","06-24-2008","TRG","121-82-4","0.325","2","0.13",,"INIT","00:00",,,"210389017","638"

知道为什么这行不通吗？同样，它在 SSMS 中运行良好，只是不是 Python/pyodbc。

Answer 1

在某些情况下，您需要在行终止符和新行一起添加一个回车return。

尝试将 \r 添加到行终止符参数。

ROWTERMINATOR = '\r\n'

由于 CSV 数据文件中的列值无效，批量加载失败

Bulk load failed due to invalid column value in CSV data file

python

sql-server

csv

pyodbc

bulkinsert