对称 DS 文件同步
SymmetricDS File Synchronisation
我正在使用 SymmetricDS（版本 3.12.3）在服务器（我们称之为 omni）和客户端（我们称之为网站）之间设置文件同步。我一直在使用 SymmetricDS 一段时间来在上述客户端和服务器之间进行数据库同步。但是，一旦 FileSyncTrackerJob 运行并检测到文件已 created/modified/deleted 在服务器上的源目录中，SymmetricDS 就会抛出错误。它抛出以下错误：
终端日志输出：
[omni-000] - FileTriggerTracker - Tracked 2 files in 0 seconds. Found 2 files changed.
[] - DataGapRouteReader - org.jumpmind.db.sql.SqlException: The result set is closed
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:306)
at org.jumpmind.db.platform.firebird.FirebirdJdbcSqlTemplate.translate(FirebirdJdbcSqlTemplate.java:62)
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:295)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:146)
at org.jumpmind.symmetric.route.DataGapRouteReader.fillPeekAheadQueue(DataGapRouteReader.java:478)
at org.jumpmind.symmetric.route.DataGapRouteReader.executeNonTransactional(DataGapRouteReader.java:250)
at org.jumpmind.symmetric.route.DataGapRouteReader.execute(DataGapRouteReader.java:155)
at org.jumpmind.symmetric.route.DataGapRouteReader.run(DataGapRouteReader.java:130)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.sql.SQLException: The result set is closed
at org.firebirdsql.jdbc.AbstractResultSet.checkOpen(AbstractResultSet.java:297)
at org.firebirdsql.jdbc.AbstractResultSet.checkCursorMove(AbstractResultSet.java:285)
at org.firebirdsql.jdbc.AbstractResultSet.next(AbstractResultSet.java:360)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:132)
... 7 more
日志文件输出有点冗长：
2020-09-02 18:25:01,427 DEBUG [] [JdbcSqlTemplate] [omni-000-router-reader-2] (20ms.) select d.data_id, d.table_name, d.event_type, cast(d.row_data as varchar(20000)) as row_data, cast(d.pk_data as varchar(1000)) as pk_data, cast(d.old_data as varchar(20000)) as old_data, d.create_time, d.trigger_hist_id, d.channel_id, d.transaction_id, d.source_node_id, d.external_data, d.node_list, d.is_prerouted from sym_data d where d.channel_id='filesync' and ((d.data_id between 15398 and 50015397)) order by d.data_id asc
2020-09-02 18:25:01,429 ERROR [] [DataGapRouteReader] [omni-000-router-reader-2] StackTraceKey.init [SqlException:994639622] org.jumpmind.db.sql.SqlException: The result set is closed
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:306)
at org.jumpmind.db.platform.firebird.FirebirdJdbcSqlTemplate.translate(FirebirdJdbcSqlTemplate.java:62)
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:295)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:146)
at org.jumpmind.symmetric.route.DataGapRouteReader.fillPeekAheadQueue(DataGapRouteReader.java:478)
at org.jumpmind.symmetric.route.DataGapRouteReader.executeNonTransactional(DataGapRouteReader.java:250)
at org.jumpmind.symmetric.route.DataGapRouteReader.execute(DataGapRouteReader.java:155)
at org.jumpmind.symmetric.route.DataGapRouteReader.run(DataGapRouteReader.java:130)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.sql.SQLException: The result set is closed
at org.firebirdsql.jdbc.AbstractResultSet.checkOpen(AbstractResultSet.java:297)
at org.firebirdsql.jdbc.AbstractResultSet.checkCursorMove(AbstractResultSet.java:285)
at org.firebirdsql.jdbc.AbstractResultSet.next(AbstractResultSet.java:360)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:132)
... 7 more
这是我的 sql 用于为文件同步设置适当的符号表：
SYM_FILE_TRIGGER table
INSERT INTO SYM_FILE_TRIGGER
( TRIGGER_ID,
CHANNEL_ID,
RELOAD_CHANNEL_ID,
BASE_DIR,
RECURSE,
INCLUDES_FILES,
EXCLUDES_FILES,
SYNC_ON_CREATE,
SYNC_ON_MODIFIED,
SYNC_ON_DELETE,
SYNC_ON_CTL_FILE,
DELETE_AFTER_SYNC,
BEFORE_COPY_SCRIPT,
AFTER_COPY_SCRIPT,
CREATE_TIME,
LAST_UPDATE_BY,
LAST_UPDATE_TIME,
DESCRIPTION
)
VALUES
( 'file_trigger',
'filesync',
'filesync_reload',
'c:\website_images\',
1,
'*.jpg',
NULL,
1,
1,
1,
0,
0,
NULL,
NULL,
current_timestamp,
'Admin',
current_timestamp,
'file trigger'
);
sym_router table
insert into sym_router
( router_id,
source_node_group_id,
target_node_group_id,
router_type,
sync_on_update,
sync_on_insert,
sync_on_delete,
create_time,
last_update_by,
last_update_time,
description
)
VALUES
( 'filerouter_omniserver_2_website',
'omni',
'website',
'default',
1,
1,
1,
current_timestamp,
'Admin',
current_timestamp,
'file router'
);
SYM_FILE_TRIGGER_ROUTER table
INSERT INTO SYM_FILE_TRIGGER_ROUTER
( TRIGGER_ID,
ROUTER_ID,
ENABLED,
INITIAL_LOAD_ENABLED,
TARGET_BASE_DIR,
CONFLICT_STRATEGY,
CREATE_TIME,
LAST_UPDATE_BY,
LAST_UPDATE_TIME,
DESCRIPTION
)
VALUES
( 'file_trigger',
'filerouter_omniserver_2_website',
1,
1,
'\home\filesync',
'source_wins',
current_timestamp,
'Admin',
current_timestamp,
'file trigger router'
);
我有 2 个符号节点组，node_group_id 分别为 omni 和 website。
omni 服务器是 Windows 机器，而客户端是 Linux (Ubuntu) 机器。仅供参考，数据库同步是在服务器上的 firebird 数据库和客户端上的 mysql 数据库之间进行的。
当使用“非事务”批处理算法时，这看起来像是 Firebird 上 SymmetricDS 可能存在的错误。文件同步通道的批处理算法设置为“非事务性”，而更改数据捕获通道的批处理算法设置为“默认”。尝试将文件同步切换为使用“默认”批处理算法，如下所示：
update sym_channel set batch_algorithm = 'default' where channel_id = 'filesync'
我正在使用 SymmetricDS（版本 3.12.3）在服务器（我们称之为 omni）和客户端（我们称之为网站）之间设置文件同步。我一直在使用 SymmetricDS 一段时间来在上述客户端和服务器之间进行数据库同步。但是，一旦 FileSyncTrackerJob 运行并检测到文件已 created/modified/deleted 在服务器上的源目录中，SymmetricDS 就会抛出错误。它抛出以下错误：
终端日志输出：
[omni-000] - FileTriggerTracker - Tracked 2 files in 0 seconds. Found 2 files changed.
[] - DataGapRouteReader - org.jumpmind.db.sql.SqlException: The result set is closed
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:306)
at org.jumpmind.db.platform.firebird.FirebirdJdbcSqlTemplate.translate(FirebirdJdbcSqlTemplate.java:62)
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:295)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:146)
at org.jumpmind.symmetric.route.DataGapRouteReader.fillPeekAheadQueue(DataGapRouteReader.java:478)
at org.jumpmind.symmetric.route.DataGapRouteReader.executeNonTransactional(DataGapRouteReader.java:250)
at org.jumpmind.symmetric.route.DataGapRouteReader.execute(DataGapRouteReader.java:155)
at org.jumpmind.symmetric.route.DataGapRouteReader.run(DataGapRouteReader.java:130)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.sql.SQLException: The result set is closed
at org.firebirdsql.jdbc.AbstractResultSet.checkOpen(AbstractResultSet.java:297)
at org.firebirdsql.jdbc.AbstractResultSet.checkCursorMove(AbstractResultSet.java:285)
at org.firebirdsql.jdbc.AbstractResultSet.next(AbstractResultSet.java:360)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:132)
... 7 more
日志文件输出有点冗长：
2020-09-02 18:25:01,427 DEBUG [] [JdbcSqlTemplate] [omni-000-router-reader-2] (20ms.) select d.data_id, d.table_name, d.event_type, cast(d.row_data as varchar(20000)) as row_data, cast(d.pk_data as varchar(1000)) as pk_data, cast(d.old_data as varchar(20000)) as old_data, d.create_time, d.trigger_hist_id, d.channel_id, d.transaction_id, d.source_node_id, d.external_data, d.node_list, d.is_prerouted from sym_data d where d.channel_id='filesync' and ((d.data_id between 15398 and 50015397)) order by d.data_id asc
2020-09-02 18:25:01,429 ERROR [] [DataGapRouteReader] [omni-000-router-reader-2] StackTraceKey.init [SqlException:994639622] org.jumpmind.db.sql.SqlException: The result set is closed
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:306)
at org.jumpmind.db.platform.firebird.FirebirdJdbcSqlTemplate.translate(FirebirdJdbcSqlTemplate.java:62)
at org.jumpmind.db.sql.AbstractSqlTemplate.translate(AbstractSqlTemplate.java:295)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:146)
at org.jumpmind.symmetric.route.DataGapRouteReader.fillPeekAheadQueue(DataGapRouteReader.java:478)
at org.jumpmind.symmetric.route.DataGapRouteReader.executeNonTransactional(DataGapRouteReader.java:250)
at org.jumpmind.symmetric.route.DataGapRouteReader.execute(DataGapRouteReader.java:155)
at org.jumpmind.symmetric.route.DataGapRouteReader.run(DataGapRouteReader.java:130)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.sql.SQLException: The result set is closed
at org.firebirdsql.jdbc.AbstractResultSet.checkOpen(AbstractResultSet.java:297)
at org.firebirdsql.jdbc.AbstractResultSet.checkCursorMove(AbstractResultSet.java:285)
at org.firebirdsql.jdbc.AbstractResultSet.next(AbstractResultSet.java:360)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
at org.jumpmind.db.sql.JdbcSqlReadCursor.next(JdbcSqlReadCursor.java:132)
... 7 more
这是我的 sql 用于为文件同步设置适当的符号表：
SYM_FILE_TRIGGER table
INSERT INTO SYM_FILE_TRIGGER
( TRIGGER_ID,
CHANNEL_ID,
RELOAD_CHANNEL_ID,
BASE_DIR,
RECURSE,
INCLUDES_FILES,
EXCLUDES_FILES,
SYNC_ON_CREATE,
SYNC_ON_MODIFIED,
SYNC_ON_DELETE,
SYNC_ON_CTL_FILE,
DELETE_AFTER_SYNC,
BEFORE_COPY_SCRIPT,
AFTER_COPY_SCRIPT,
CREATE_TIME,
LAST_UPDATE_BY,
LAST_UPDATE_TIME,
DESCRIPTION
)
VALUES
( 'file_trigger',
'filesync',
'filesync_reload',
'c:\website_images\',
1,
'*.jpg',
NULL,
1,
1,
1,
0,
0,
NULL,
NULL,
current_timestamp,
'Admin',
current_timestamp,
'file trigger'
);
sym_router table
insert into sym_router
( router_id,
source_node_group_id,
target_node_group_id,
router_type,
sync_on_update,
sync_on_insert,
sync_on_delete,
create_time,
last_update_by,
last_update_time,
description
)
VALUES
( 'filerouter_omniserver_2_website',
'omni',
'website',
'default',
1,
1,
1,
current_timestamp,
'Admin',
current_timestamp,
'file router'
);
SYM_FILE_TRIGGER_ROUTER table
INSERT INTO SYM_FILE_TRIGGER_ROUTER
( TRIGGER_ID,
ROUTER_ID,
ENABLED,
INITIAL_LOAD_ENABLED,
TARGET_BASE_DIR,
CONFLICT_STRATEGY,
CREATE_TIME,
LAST_UPDATE_BY,
LAST_UPDATE_TIME,
DESCRIPTION
)
VALUES
( 'file_trigger',
'filerouter_omniserver_2_website',
1,
1,
'\home\filesync',
'source_wins',
current_timestamp,
'Admin',
current_timestamp,
'file trigger router'
);
我有 2 个符号节点组，node_group_id 分别为 omni 和 website。
omni 服务器是 Windows 机器，而客户端是 Linux (Ubuntu) 机器。仅供参考，数据库同步是在服务器上的 firebird 数据库和客户端上的 mysql 数据库之间进行的。
当使用“非事务”批处理算法时，这看起来像是 Firebird 上 SymmetricDS 可能存在的错误。文件同步通道的批处理算法设置为“非事务性”，而更改数据捕获通道的批处理算法设置为“默认”。尝试将文件同步切换为使用“默认”批处理算法，如下所示：
update sym_channel set batch_algorithm = 'default' where channel_id = 'filesync'