Talend ETL - mySQLInput 组件

Question

只是在寻找一些关于 Talend ETL 的一般性建议。

我是 Talend ETL 的新手，但不是一般的 ETL 工具。我创建了一个复杂的 MySQL 查询，它有几个子查询，其中 return 来自其他 table 的数据，以及几个 case 语句。我希望使用 MySQLInput 组件来导入数据并使用 tmap 以我想要的方式输出数据。

查询在 MySQL workbench 中完美运行。但是，当我在 talend 中使用它时，它会抛出很多错误，我不知道从哪里开始。

MySQL输入组件真的只允许从一个table中提取数据吗？这似乎是毫无意义的限制。

如果有人想看，我可以post查询。

谢谢，尼克。

查询：

"select     
LA.ENCODEDKEY
,LA.ID
,LA.ACCOUNTSTATE
,LA.RESCHEDULEDACCOUNTKEY
,LA.CLOSEDDATE
,LA.LASTLOCKEDDATE
,LA.CREATIONDATE
,LA.APPROVEDDATE
,DATE_FORMAT(LA.DISBURSEMENTDATE,"%Y%m%d") AS DISBURESEMENTDATE
,DATE_FORMAT(LA.EXPECTEDDISBURSEMENTDATE,"%Y%m%d") AS EXPECTEDDISBURESEMENTDATE
,LA.INTERESTRATE
,LA.LOANAMOUNT
,ROUND(LA.INTERESTBALANCE + LA.PRINCIPALBALANCE + LA.ACCRUEDINTEREST + LA.FEESBALANCE + LA.PENALTYBALANCE,0) AS ACCOUNTINGBALANCE
,LA.PRINCIPALBALANCE AS INTERESTBALANCE
,CASE WHEN LA.REPAYMENTPERIODCOUNT = 3 THEN 4 ELSE 12 END AS REPAYMENTFREQUENCY
,ARR.ARREARSDATE
,CASE WHEN ARR.ARREARSAMOUNT IS NULL THEN 0 ELSE ARR.ARREARSAMOUNT END AS ARREARSAMOUNT
,MATD.DUEDATE
,LA.ACCRUEDINTEREST
,LA.FEESBALANCE
,LA.PENALTYBALANCE
,LA.INTERESTBALANCE AS MAMBUINTERESTBALANCE
,CASE WHEN(SELECT cv.`VALUE` as vl FROM customfieldvalue cv     JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'ON_Loan_Offer_Expiry_Date') IS NULL THEN DATE_FORMAT(NOW(),"%Y-%m-%d") ELSE(SELECT cv.`VALUE` as vl FROM customfieldvalue cv    JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'ON_Loan_Offer_Expiry_Date') END AS LOANOFFEREXPIRTYDATE
,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv  JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'ON_DD_Manual_Offline_Schedule') AS MANUALOFFLINE
,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv  JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'ON_Offer_Docs_Received') AS OFFERDOCSRECEIVED
,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv  JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'Security/Collateral written down Value') AS SECURITY
,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv  JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = 'Actual Provision') AS SPECIFICPROVISION
,DATEDIFF(MATD.DUEDATE,NOW()) AS LOANMATURITYDAYS
,CASE WHEN (LA.ACCRUEDINTEREST + LA.FEESBALANCE + LA.PENALTYBALANCE + LA.INTERESTBALANCE) = 0 THEN 1 ELSE (LA.ACCRUEDINTEREST + LA.FEESBALANCE + LA.PENALTYBALANCE + LA.INTERESTBALANCE) END AS LOANACCOUNTADJUST
FROM
LOANACCOUNT AS LA
LEFT join
(SELECT
STATE
,PARENTACCOUNTKEY
,MIN(DUEDATE) AS ARREARSDATE
,SUM(INTERESTDUE - INTERESTPAID + PRINCIPALDUE - PRINCIPALPAID + FEESDUE - FEESPAID + PENALTYDUE - PENALTYPAID) AS ARREARSAMOUNT
FROM
REPAYMENT
WHERE STATE = "LATE"
GROUP BY STATE,PARENTACCOUNTKEY) AS ARR ON ARR.PARENTACCOUNTKEY = LA.ENCODEDKEY
left join
(SELECT
PARENTACCOUNTKEY
,MAX(DUEDATE) AS DUEDATE
FROM
REPAYMENT
GROUP BY PARENTACCOUNTKEY) AS MATD ON MATD.PARENTACCOUNTKEY = LA.ENCODEDKEY"

完整的错误信息：

    Syntax error on token "") ELSE(SELECT cv.`VALUE` as vl FROM customfieldvalue cv     JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = \'ON_Loan_Offer_Expiry_Date\') END AS LOANOFFEREXPIRTYDATE\r\n,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv     JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = \'ON_DD_Manual_Offline_Schedule\') AS MANUALOFFLINE\r\n,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv    JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = \'ON_Offer_Docs_Received\') AS OFFERDOCSRECEIVED\r\n,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv   JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = \'Security/Collateral written down Value\') AS SECURITY\r\n,(SELECT cv.`VALUE` as vl FROM customfieldvalue cv    JOIN customfield cf on cv.CUSTOMFIELDKEY = cf.ENCODEDKEY    WHERE LA.ENCODEDKEY = cv.PARENTKEY AND cf.ID = \'Actual Provision\') AS SPECIFICPROVISION\r\n,DATEDIFF(MATD.DUEDATE,NOW()) AS LOANMATURITYDAYS\r\n,CASE WHEN (LA.ACCRUEDINTEREST + LA.FEESBALANCE + LA.PENALTYBALANCE + LA.INTERESTBALANCE) = 0 THEN 1 ELSE (LA.ACCRUEDINTEREST + LA.FEESBALANCE + LA.PENALTYBALANCE + LA.INTERESTBALANCE) END AS LOANACCOUNTADJUST\r\nFROM\r\nLOANACCOUNT AS LA\r\nLEFT join\r\n(SELECT\r\nSTATE\r\n,PARENTACCOUNTKEY\r\n,MIN(DUEDATE) AS ARREARSDATE\r\n,SUM(INTERESTDUE - INTERESTPAID + PRINCIPALDUE - PRINCIPALPAID + FEESDUE - FEESPAID + PENALTYDUE - PENALTYPAID) AS ARREARSAMOUNT\r\nFROM\r\nREPAYMENT\r\nWHERE STATE = \'LATE\'\r\nGROUP BY STATE,PARENTACCOUNTKEY) AS ARR ON ARR.PARENTACCOUNTKEY = LA.ENCODEDKEY\r\nleft join\r\n(SELECT\r\nPARENTACCOUNTKEY\r\n,MAX(DUEDATE) AS DUEDATE\r\nFROM\r\nREPAYMENT\r\nGROUP BY PARENTACCOUNTKEY) AS MATD ON MATD.PARENTACCOUNTKEY = LA.ENCODEDKEY\r\n"", delete this token

Answer 1

您可以在 tMySQLInput 组件中放置任何 select 查询，如果您的查询读取很多 table 则让字段 table 名称像这样空白 "" 并且它将工作。你的查询应该在两个 " 之内并且没有空行，因为它是一个字符串，你必须使用 ' 而不是 " 或者如果不可能的话转义它：

..
WHERE STATE = 'LATE'
..
DATE_FORMAT(LA.DISBURSEMENTDATE,'%Y%m%d')
..

Answer 2

仅供参考，如果其他人遇到此问题。

我的问题似乎是我在计算字段中留下了空格，即 la.feespaid + la.penaltypaid

需要一行，有空格。所以：la.feespaid+la.penaltypaid

实际上，我想我了解到 Talend 与 informatica 非常相似，对代码中的 'grammar' 非常敏感。

Talend ETL - mySQLInput 组件

Talend ETL - mySQLInput component

mysql

etl

talend