Teradata - 翻译过程中遇到非法形成的字符串
Teradata - An illegally formed character string was encountered during translation
我正在通过 Twitter API 在 pandas 数据框中获取推文,并将数据写入 teradata 数据库。然而,与其他推文不同的是,一个单元格具有包含粗体数据的特定推文。当我尝试将其插入数据库时,弹出以下错误:
OperationalError: [Version 17.0.0.4] [Session 3046127] [Teradata SQL Driver] [Error 528] A failure occurred while executing rows 1 through 292 of a batch request.
at gosqldriver/teradatasql.(*teradataConnection).makeDriverErrorCode TeradataConnection.go:1120
at gosqldriver/teradatasql.newTeradataRows TeradataRows.go:396
at gosqldriver/teradatasql.(*teradataStatement).QueryContext TeradataStatement.go:122
at gosqldriver/teradatasql.(*teradataConnection).QueryContext TeradataConnection.go:2083
at database/sql.ctxDriverQuery ctxutil.go:48
at database/sql.(*DB).queryDC.func1 sql.go:1579
at database/sql.withLock sql.go:3204
at database/sql.(*DB).queryDC sql.go:1574
at database/sql.(*Conn).QueryContext sql.go:1823
at main.goCreateRows goside.go:654
at main._cgoexpwrap_cfa80c8a3acb_goCreateRows _cgo_gotypes.go:363
at runtime.cgocallbackg1 cgocall.go:332
at runtime.cgocallbackg cgocall.go:207
at runtime.cgocallback_gofunc asm_amd64.s:793
at runtime.goexit asm_amd64.s:1373
Caused by [Version 17.0.0.4] [Session 3046127] [Teradata Database] [Error 6705] An illegally formed character string was encountered during translation.
at gosqldriver/teradatasql.(*teradataConnection).formatDatabaseError TeradataConnection.go:1138
at gosqldriver/teradatasql.(*teradataConnection).makeChainedDatabaseError TeradataConnection.go:1154
数据库中的推文数据类型是“varchar(1000) CHARACTER SET UNICODE NOT CASESPECIFIC”
示例数据如下:
包含粗体文本的推文导致插入出现问题。我该如何缓解这种情况?
要存储或检索任意 Unicode 代码点,请使用 Unicode 传递功能来加载和查询会话。
SET SESSION CHARACTER SET UNICODE PASS THROUGH ON;
对于给出的具体示例,您可能会发现“规范化”Unicode 文本很有用,例如如果您想要相应的 ASCII 字母字符,则在加载前使用 Python unicodedata.normalize
或在加载后使用 Teradata TRANSLATE(...)
- 但这不适用于其他 Unicode 字符,例如输入中也可能出现的表情符号.
我正在通过 Twitter API 在 pandas 数据框中获取推文,并将数据写入 teradata 数据库。然而,与其他推文不同的是,一个单元格具有包含粗体数据的特定推文。当我尝试将其插入数据库时,弹出以下错误:
OperationalError: [Version 17.0.0.4] [Session 3046127] [Teradata SQL Driver] [Error 528] A failure occurred while executing rows 1 through 292 of a batch request.
at gosqldriver/teradatasql.(*teradataConnection).makeDriverErrorCode TeradataConnection.go:1120
at gosqldriver/teradatasql.newTeradataRows TeradataRows.go:396
at gosqldriver/teradatasql.(*teradataStatement).QueryContext TeradataStatement.go:122
at gosqldriver/teradatasql.(*teradataConnection).QueryContext TeradataConnection.go:2083
at database/sql.ctxDriverQuery ctxutil.go:48
at database/sql.(*DB).queryDC.func1 sql.go:1579
at database/sql.withLock sql.go:3204
at database/sql.(*DB).queryDC sql.go:1574
at database/sql.(*Conn).QueryContext sql.go:1823
at main.goCreateRows goside.go:654
at main._cgoexpwrap_cfa80c8a3acb_goCreateRows _cgo_gotypes.go:363
at runtime.cgocallbackg1 cgocall.go:332
at runtime.cgocallbackg cgocall.go:207
at runtime.cgocallback_gofunc asm_amd64.s:793
at runtime.goexit asm_amd64.s:1373
Caused by [Version 17.0.0.4] [Session 3046127] [Teradata Database] [Error 6705] An illegally formed character string was encountered during translation.
at gosqldriver/teradatasql.(*teradataConnection).formatDatabaseError TeradataConnection.go:1138
at gosqldriver/teradatasql.(*teradataConnection).makeChainedDatabaseError TeradataConnection.go:1154
数据库中的推文数据类型是“varchar(1000) CHARACTER SET UNICODE NOT CASESPECIFIC”
示例数据如下:
包含粗体文本的推文导致插入出现问题。我该如何缓解这种情况?
要存储或检索任意 Unicode 代码点,请使用 Unicode 传递功能来加载和查询会话。
SET SESSION CHARACTER SET UNICODE PASS THROUGH ON;
对于给出的具体示例,您可能会发现“规范化”Unicode 文本很有用,例如如果您想要相应的 ASCII 字母字符,则在加载前使用 Python unicodedata.normalize
或在加载后使用 Teradata TRANSLATE(...)
- 但这不适用于其他 Unicode 字符,例如输入中也可能出现的表情符号.