Pentaho数据集成如何运行 job with kitchen on carte cluster?

Pentaho Data Integration How to run job with kitchen on carte cluster?

我已经建立了一个 carte 集群(1 个主控和 2 个从属)和 运行 用勺子在 carte 集群上工作。但是当我 运行 使用 kitchen 命令或 carte http 访问时,它 运行 作为独立的(只是 运行 在主节点中)。

我是否遗漏了配置中的任何内容?还是不支持集群模式?

这是我尝试过的:

  1. 我的配置:

  1. 运行 用勺子 "Enviroment Type -- Local"

    主输出:

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
    Tue Nov 28 04:47:09 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    2017/11/28 04:47:10 - sortcluster111 (master) - Dispatching started for transformation [sortcluster111 (master)]
    Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    Tue Nov 28 04:47:10 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must   be       established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to   explicitly disable SSL       by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    2017/11/28 04:47:10 - output.0 - Connected to database [102] (commit=1000)
    2017/11/28 04:47:10 - input.0 - Finished reading query, closing connection.
    2017/11/28 04:47:10 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
    2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40001], reading from server Dynamic slave [kettleslave02:8083]
    2017/11/28 04:47:10 - input.0 - Server socket accepted for port [40000], reading from server Dynamic slave [kettleslave01:8082]
    2017/11/28 04:47:10 - output.0 - Finished processing (I=47, O=47, R=0, W=47, U=0, E=0)      
    

    slave01输出:

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
    Tue Nov 28 04:47:09 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must  be        established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to  explicitly disable SSL        by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    2017/11/28 04:47:10 - sortcluster111 (cluster1:Dynamic slave [kettleslave01:8082]) - Dispatching started for transformation [sortcluster111 (cluster1:Dynamic slave [kettleslave01:8082])]
    2017/11/28 04:47:10 - sort.0 - Server socket accepted for port [40000], reading from server kettlemaster01
    2017/11/28 04:47:10 - sort.0 - Finished processing (I=24, O=0, R=0, W=24, U=0, E=0)
    

    slave02输出:

    2017/11/28 04:47:09 - RepositoriesMeta - Reading repositories XML file: /root/.kettle/repositories.xml
    2017/11/28 04:47:09 - General - Unable to connect to the repository with name 'Mysqlrep'
    2017/11/28 04:47:10 - sortcluster111 (cluster1:Dynamic slave [kettleslave02:8083]) - Dispatching started for transformation [sortcluster111 (cluster1:Dynamic slave [kettleslave02:8083])]
    2017/11/28 04:47:10 - sort.0 - Server socket accepted for port [40000], reading from server kettlemaster01
    2017/11/28 04:47:10 - sort.0 - Finished processing (I=23, O=0, R=0, W=23, U=0, E=0)        
    
  2. 运行 带厨房:

    kitchen.sh -rep=Mysqlrep -user=admin -pass=admin -job trans1
    

    主输出:

    2017/11/28 04:10:19 - trans1 - Starting entry [sorttrans]
    2017/11/28 04:10:19 - sorttrans - Loading transformation from repository [sortcluster111] in directory [/]
    2017/11/28 04:10:19 - sorttrans - Using run configuration [cluster config]
    2017/11/28 04:10:19 - sorttrans - Using legacy execution engine
    2017/11/28 04:10:19 - sortcluster111 - Dispatching started for transformation [sortcluster111]
    Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    Tue Nov 28 04:10:19 EST 2017 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be         established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL         by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
    2017/11/28 04:10:19 - output.0 - Connected to database [102] (commit=1000)
    2017/11/28 04:10:19 - input.0 - Finished reading query, closing connection.
    2017/11/28 04:10:19 - input.0 - Finished processing (I=47, O=0, R=0, W=47, U=0, E=0)
    2017/11/28 04:10:19 - sort.0 - Finished processing (I=0, O=0, R=47, W=47, U=0, E=0)
    2017/11/28 04:10:19 - output.0 - Finished processing (I=0, O=47, R=47, W=47, U=0, E=0)
    2017/11/28 04:10:19 - trans1 - Starting entry [finish]
    2017/11/28 04:10:19 - trans1 - Finished job entry [finish] (result=[true])
    2017/11/28 04:10:19 - trans1 - Finished job entry [sorttrans] (result=[true])
    2017/11/28 04:10:19 - trans1 - Finished job entry [SQL] (result=[true])
    2017/11/28 04:10:19 - trans1 - Job execution finished
    2017/11/28 04:10:19 - Kitchen - Finished!
    2017/11/28 04:10:19 - Kitchen - Start=2017/11/28 04:10:00.586, Stop=2017/11/28 04:10:19.739
    2017/11/28 04:10:19 - Kitchen - Processing ended after 19 seconds.
    

    slave中无输出

此致

约翰

首先,在远程服务器上开始点菜:

  • ./carte.sh hostname port
  • 检查您的浏览器:hostname:port/kettle/status/ 返回一个空的状态页面

其次,在spoon中定义carte:

  • 在勺子视图(左面板)中:单击 Slave server,然后单击 new
  • 服务器名称=点菜
  • hostname = 先前定义的主机名
  • port = 先前定义的端口
  • 网络应用名称=空白(重要!)
  • 用户名=集群
  • 密码=集群 是大师=

三、配置勺子运行:

  • 在勺子视图(左面板)中:单击 Run configuration,然后单击 new
  • 姓名:carte/servername
  • 引擎:Pentaho(默认)
  • 从属服务器:是
  • 位置:carte(第二步中定义的服务器名称)
  • 向此服务器发送资源:是(除非你read the doc

四、运行 transformaton/job

  • 运行 配置下拉框让您选择 Pentaho localcarte/hostname,或第三步中定义的任何配置。
  • 如果您禁用了 Run option 面板,则下拉 运行 工具或顶部 menu/Action/Run 选项或 F8。并勾选 Always show dialog on run,你以后会用到它。

第五名

  • 在您的浏览器中:hostname:port/kettle/status/
  • 玩得开心。

新版pdi存在一个bug: 选项 "Run this transformation in a clustered mode?" 不存在,因此要解决此问题,您需要打开作业 xml 文件并删除 属性 run_configuration 并设置集群 到 Y 从您想要在集群模式下 运行 的转换条目。 希望这有帮助。