Informatica CDC 拒绝数据
Informatica CDC rejects data
我们的源是安装了 Informatica CDC 的 oracle ERP 系统，我们的目标是 Greenplum tables，我们将数据作为 1-1 逻辑加载到其中。我们以实时模式执行会话，这意味着会话将保持 运行，当源发生任何更改时，会话将处理并反映在目标 table.
我们收到以下错误数据拒绝。
2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS |
INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8229 : Database errors
occurred:
FnName: Execute -- [Informatica][ODBC Greenplum Wire Protocol driver]Socket closed. 2015-09-15 12:31:42 : ERROR : (22994 |
WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8425
: ERROR: Writer execution failed. 2015-09-15 12:31:42 : ERROR : (22994
| WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 :
WRT_8164 : Error loading into target [ods_oe_sales_credits] :
2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS |
INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8114 :
Row # [1] in bad file
2015-09-15 12:31:42 : INFO : (22994 | WRITER_1_*_1) : (IS |
INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : CMN_1053 : : Rowdata: (
RowType=1(update) Src Rowid=1 Targ Rowid=1 sales_credit_id
(sales_credit_id:BigInt:): "19094" creation_date
(creation_date:Date:): "09/15/2015 03:01:27.000000000" created_by
(created_by:Double:): "1193.000000000000" last_update_date
(last_update_date:Date:): "09/15/2015 03:01:27.000000000"
last_updated_by (last_updated_by:Double:): "1193.000000000000"
last_update_login (last_update_login:Double:): "240638.0000000000"
header_id (header_id:BigInt:): "27535" salesrep_id
(salesrep_id:BigInt:): "-3" percent (percent:Double:):
"100.0000000000000" line_id (line_id:BigInt:): "(NULL)" context
(context:Char.60:): "(NULL)" attribute1 (attribute1:Char.480:):
"(NULL)" attribute2 (attribute2:Char.480:): "(NULL)" attribute3
(attribute3:Char.480:): "(NULL)" attribute4 (attribute4:Char.480:):
"(NULL)" attribute5 (attribute5:Char.480:): "(NULL)" attribute6
(attribute6:Char.480:): "(NULL)" attribute7 (attribute7:Char.480:):
"(NULL)" attribute8 (attribute8:Char.480:): "(NULL)" attribute9
(attribute9:Char.480:): "(NULL)" attribute10
(attribute10:Char.480:): "(NULL)" attribute11
(attribute11:Char.480:): "(NULL)" attribute12
(attribute12:Char.480:): "(NULL)" attribute13
(attribute13:Char.480:): "(NULL)" attribute14
(attribute14:Char.480:): "(NULL)" attribute15
(attribute15:Char.480:): "(NULL)" dw_update_advice_flag
(dw_update_advice_flag:Char.2:): "(NULL)" wh_update_date
(wh_update_date:Date:): "(NULL)" orig_sys_credit_ref
(orig_sys_credit_ref:Char.100:): "OE_SALES_CREDITS19094"
sales_credit_type_id (sales_credit_type_id:BigInt:): "1"
lock_control (lock_control:Double:): "1.000000000000000"
sales_group_id (sales_group_id:BigInt:): "-1"
sales_group_updated_flag (sales_group_updated_flag:Char.2:): "(NULL)"
inst_id (inst_id:BigInt:): "(NULL)" md_source_system
(md_source_system:Double:): "1.000000000000000" flag (flag:Char.2:):
"I" md_deleted_flag (md_deleted_flag:Char.2:): "(NULL)" change_ts
(change_ts:Date:): "09/15/2015 03:01:42.000000000" )
2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS |
INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8164 : Error loading into
target [ods_oe_sales_credits] : Bad rows exceeded Session Threshold
[1] 2015-09-15 12:31:42 : INFO : (22994 | WRITER_1__1) : (IS |
INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8333 : Rolling back all
the targets due to fatal session error. 2015-09-15 12:31:42 : ERROR :
(22994 | WRITER_1_*_1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 :
CMN_1022 : Database driver error... CMN_1022 []
不确定实际问题出在哪里（informatica/DB 一方）当检查错误文件时它说记录因 NULL 问题而被拒绝
错误的文件内容：
1,D,19094,D,09/15/2015 03:01:27.000000000,D,1193,D,09/15/2015
03:01:27.000000000,D,1193,D,240638,D,27535,D,-3,D,100,D,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,OE_SALES_CREDITS19094,D,1,D,1,D,-1,D,,N,,N,1,D,I,D,,N,09/15/2015
03:01:42.000000000,D
但是同一个会话处理其他具有空值的记录。无法理解问题到底出在哪里。
请在这方面帮助我...
我们找到了解决方案。 GP 防火墙设置是导致 运行 长查询失败的罪魁祸首。 GP DBA 在数据库级别覆盖了此设置，现在我们所有的 CDC 会话 运行 连续几天没有失败。
我们的源是安装了 Informatica CDC 的 oracle ERP 系统，我们的目标是 Greenplum tables，我们将数据作为 1-1 逻辑加载到其中。我们以实时模式执行会话，这意味着会话将保持 运行，当源发生任何更改时，会话将处理并反映在目标 table.
我们收到以下错误数据拒绝。
2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8229 : Database errors occurred: FnName: Execute -- [Informatica][ODBC Greenplum Wire Protocol driver]Socket closed. 2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8425 : ERROR: Writer execution failed. 2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8164 : Error loading into target [ods_oe_sales_credits] : 2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8114 : Row # [1] in bad file
2015-09-15 12:31:42 : INFO : (22994 | WRITER_1_*_1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : CMN_1053 : : Rowdata: ( RowType=1(update) Src Rowid=1 Targ Rowid=1 sales_credit_id (sales_credit_id:BigInt:): "19094" creation_date (creation_date:Date:): "09/15/2015 03:01:27.000000000" created_by (created_by:Double:): "1193.000000000000" last_update_date (last_update_date:Date:): "09/15/2015 03:01:27.000000000"
last_updated_by (last_updated_by:Double:): "1193.000000000000"
last_update_login (last_update_login:Double:): "240638.0000000000"
header_id (header_id:BigInt:): "27535" salesrep_id (salesrep_id:BigInt:): "-3" percent (percent:Double:): "100.0000000000000" line_id (line_id:BigInt:): "(NULL)" context (context:Char.60:): "(NULL)" attribute1 (attribute1:Char.480:): "(NULL)" attribute2 (attribute2:Char.480:): "(NULL)" attribute3 (attribute3:Char.480:): "(NULL)" attribute4 (attribute4:Char.480:): "(NULL)" attribute5 (attribute5:Char.480:): "(NULL)" attribute6 (attribute6:Char.480:): "(NULL)" attribute7 (attribute7:Char.480:): "(NULL)" attribute8 (attribute8:Char.480:): "(NULL)" attribute9 (attribute9:Char.480:): "(NULL)" attribute10 (attribute10:Char.480:): "(NULL)" attribute11 (attribute11:Char.480:): "(NULL)" attribute12 (attribute12:Char.480:): "(NULL)" attribute13 (attribute13:Char.480:): "(NULL)" attribute14 (attribute14:Char.480:): "(NULL)" attribute15 (attribute15:Char.480:): "(NULL)" dw_update_advice_flag (dw_update_advice_flag:Char.2:): "(NULL)" wh_update_date (wh_update_date:Date:): "(NULL)" orig_sys_credit_ref (orig_sys_credit_ref:Char.100:): "OE_SALES_CREDITS19094"
sales_credit_type_id (sales_credit_type_id:BigInt:): "1"
lock_control (lock_control:Double:): "1.000000000000000"
sales_group_id (sales_group_id:BigInt:): "-1"
sales_group_updated_flag (sales_group_updated_flag:Char.2:): "(NULL)" inst_id (inst_id:BigInt:): "(NULL)" md_source_system (md_source_system:Double:): "1.000000000000000" flag (flag:Char.2:): "I" md_deleted_flag (md_deleted_flag:Char.2:): "(NULL)" change_ts (change_ts:Date:): "09/15/2015 03:01:42.000000000" )2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8164 : Error loading into target [ods_oe_sales_credits] : Bad rows exceeded Session Threshold [1] 2015-09-15 12:31:42 : INFO : (22994 | WRITER_1__1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : WRT_8333 : Rolling back all the targets due to fatal session error. 2015-09-15 12:31:42 : ERROR : (22994 | WRITER_1_*_1) : (IS | INF_EN1_ASCII) : NODE_ENDEV_TNLD07879 : CMN_1022 : Database driver error... CMN_1022 []
不确定实际问题出在哪里（informatica/DB 一方）当检查错误文件时它说记录因 NULL 问题而被拒绝
错误的文件内容：
1,D,19094,D,09/15/2015 03:01:27.000000000,D,1193,D,09/15/2015 03:01:27.000000000,D,1193,D,240638,D,27535,D,-3,D,100,D,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,,N,OE_SALES_CREDITS19094,D,1,D,1,D,-1,D,,N,,N,1,D,I,D,,N,09/15/2015 03:01:42.000000000,D
但是同一个会话处理其他具有空值的记录。无法理解问题到底出在哪里。
请在这方面帮助我...
我们找到了解决方案。 GP 防火墙设置是导致 运行 长查询失败的罪魁祸首。 GP DBA 在数据库级别覆盖了此设置，现在我们所有的 CDC 会话 运行 连续几天没有失败。