JDBC 批量更新的所有行的原子锁定

Atomic locking of all rows of a JDBC batch update

我有两个线程 运行 并发更新 table 类似于:

CREATE TABLE T (
  SEQ NUMBER(10) PRIMARY KEY,
  VAL1 VARCHAR2(10),
  VAL2 VARCHAR2(10)
)

table 包含大量条目,其中更新类似于:

UPDATE T SET VAL1 = ? WHERE SEQ < ?
UPDATE T SET VAL2 = ? WHERE SEQ = ?

两个语句都是 运行 在两个不同的事务中,因为 JDBC 批量更新每个 1000 行。这样做时,我很快遇到了 ORA-00060: deadlock detected while waiting for resource。我假设这两个事务都会部分影响相同的行,其中两个事务设法在另一行之前锁定一些行。

有没有办法通过使锁定原子化来避免这种情况,或者我是否需要在两个线程之间引入某种形式的显式锁定?

在这种情况下,如果您的线程无法控制不重叠数据,那么唯一的解决方案就是锁定整个 table,这不是一个很好的解决方案,因为另一个线程 (或在 table 上执行 DML 的任何其他操作都会挂起,直到锁定会话提交或回滚。您可以尝试的另一件事是让 "smaller" 人(更新单行的人)更频繁地提交(可能每个 row/execution),从而允许死锁(或锁定等待)情况可能发生的频率较低。这对 "smaller" 家伙有性能副作用。

控制你的猴子!

-吉姆

当您更新记录时,会采取锁定措施以防止会损害原子性的脏写。

但是,对于您的情况,您可以使用 SKIP LOCKED。这样,在尝试进行更新之前,您会尝试使用 SKIP LOCKED 获取 FOR UPDATE 锁。这将允许您锁定您计划修改的记录,并跳过那些已经被其他并发事务锁定的记录。

查看我的高性能 Java 持久性 GitHub 存储库中的 SkipLockJobQueueTest,了解如何使用 SKIP LOCKED 的示例。

I assume that both transaction would partially affect the same rows where both transactions managed to lock some rows before the other one.

没错。我可以建议两个选项来避免这种情况:

1)更新前使用SELECT ... FOR UPDATE子句:

SELECT * FROM T WHERE SEQ < ? FOR UPDATE;
UPDATE T SET VAL1 = ? WHERE SEQ < ?

SELECT * FROM T WHERE SEQ = ? FOR UPDATE;
UPDATE T SET VAL2 = ? WHERE SEQ = ?

谓词必须相同才能影响相同的行。
FOR UPDATE 子句使 Oracle 锁定请求的行。只要另一个会话也对 SELECT 使用 FOR UPDATE 子句,它就会被阻塞,直到前一个事务返回 committed\rolled。

2) 使用DBMS_LOCK 包来创建和控制自定义锁。获取和释放锁必须手动执行。

我找到了一个解决方案,它需要在插入端进行一些重新设计,但基本上仍然和以前一样做同样的事情。我已将 table 拆分为两个 table:

CREATE TABLE T1 (
  SEQ NUMBER(10) PRIMARY KEY,
  VAL1 VARCHAR2(10)
);

CREATE TABLE T2 (
  SEQ NUMBER(10) PRIMARY KEY,
  VAL2 VARCHAR2(10)
);

现在我可以在不锁定同一行的情况下更新列,在某种程度上我正在模拟列锁。这当然是一个重大变化,但幸运的是 Oracle 允许定义一个物化视图以避免更改任何选择:

CREATE MATERIALIZED VIEW LOG ON T1 WITH ROWID INCLUDING NEW VALUES;
CREATE MATERIALIZED VIEW LOG ON T2 WITH ROWID INCLUDING NEW VALUES;

CREATE MATERIALIZED VIEW T 
REFRESH FAST ON COMMIT
AS
SELECT SEQ, VAL1, VAL2, T1.ROWID AS T1_ROWID, T2.ROWID AS T2_ROWID
FROM T1
NATURAL JOIN T2;

这样做,我能够保留基数 table T 上的所有索引,这些索引通常同时包含 VAL1VAL2.

在此之前,我能够通过按给定顺序(从最高 SEQ 到最低)应用批量更新来大幅减少死锁数量。因此,Oracle 似乎经常使用索引顺序来锁定 table,但这也不是 100% 可靠的。

一个简单的解决方案是在共享模式下锁定 table,以确保在最大更新之前没有并发写入,使用 LOCK TABLE ... IN SHARE MODE。

如果你想重现,这是我的两个脚本: 主要的创建 table 和 运行s 测试用例 - /tmp/sql1.sql:

set echo on time on define off sqlprompt "SQL1> " linesize 69 pagesize 1000
set sqlformat ansiconsole
connect sys/oracle@//localhost/PDB1 as sysdba
grant dba to scott identified by tiger;
connect scott/tiger@//localhost/PDB1
exec begin execute immediate 'drop table T'; exception when others then null; end;
CREATE TABLE T (
  SEQ NUMBER(10) constraint T_SEQ PRIMARY KEY,
  VAL1 VARCHAR2(10),
  VAL2 VARCHAR2(10)
);
insert into T select rownum , 0 , 0 from xmltable('1 to 5');
commit;
-- -------- start session 1
connect scott/tiger@//localhost/PDB1
select sys_context('userenv','sid') from dual;
variable val number
variable seq number;
exec :seq:=4; :val:=2;
UPDATE T SET VAL2 = :val WHERE SEQ = :seq;
-- -------- call session 2
host sql /nolog @/tmp/sql2.sql < /dev/null & :
host sleep 5
select session_id,lock_type,mode_held,mode_requested,lock_id1,lock_id2,blocking_others from dba_locks where lock_type in ('DML','Transaction','PL/SQL User Lock');
-- -------- continue session 1 while session 2 waits
exec :seq:=1; :val:=3;
UPDATE T SET VAL2 = :val WHERE SEQ = :seq;
host sleep 1
commit;
select * from T;
-- -------- end session 1

第二个在主中被调用为运行同时- /tmp/sql2.sql:

set echo on time on define off sqlprompt "SQL2> "
-- -------- start session 2 -------- --
host sleep 1
connect scott/tiger@//localhost/PDB1
select sys_context('userenv','sid') from dual;
variable val number
variable seq number;
exec :seq:=5; :val:=1;
/* TM lock solution */ lock table T in share mode;
UPDATE T SET VAL1 = :val WHERE SEQ < :seq;
commit;
select * from T;
-- -------- end session 2

这是带有共享锁的 运行,我们看到 DML 锁 'Share' 被 'Row-X' 阻塞(这是更新自动获取的锁):

SQLcl: Release 18.4 Production on Wed Apr 17 09:32:04 2019

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

SQL>
SQL> set echo on time on define off sqlprompt "SQL1> " linesize 69 pagesize 1000
09:32:04 SQL1> set sqlformat ansiconsole
09:32:04 SQL1> connect sys/oracle@//localhost/PDB1 as sysdba
Connected.
09:32:05 SQL1>
09:32:05 SQL1> grant dba to scott identified by tiger;

Grant succeeded.

09:32:05 SQL1> connect scott/tiger@//localhost/PDB1
Connected.
09:32:08 SQL1>
09:32:08 SQL1> exec begin execute immediate 'drop table T'; exception when others then null; end;

PL/SQL procedure successfully completed.

09:32:09 SQL1> CREATE TABLE T (
  2    SEQ NUMBER(10) constraint T_SEQ PRIMARY KEY,
  3    VAL1 VARCHAR2(10),
  4    VAL2 VARCHAR2(10)
  5  );

Table created.

09:32:09 SQL1> insert into T select rownum , 0 , 0 from xmltable('1 to 5');

5 rows created.

09:32:09 SQL1> commit;

Commit complete.

09:32:09 SQL1> -- -------- start session 1
09:32:09 SQL1> connect scott/tiger@//localhost/PDB1
Connected.
09:32:09 SQL1>
09:32:09 SQL1> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
4479


09:32:09 SQL1> variable val number
09:32:09 SQL1> variable seq number;
09:32:09 SQL1> exec :seq:=4; :val:=2;

PL/SQL procedure successfully completed.

09:32:09 SQL1> UPDATE T SET VAL2 = :val WHERE SEQ = :seq;

1 row updated.

09:32:09 SQL1> -- -------- call session 2
09:32:09 SQL1> host sql /nolog @/tmp/sql2.sql < /dev/null & :

09:32:09 SQL1> host sleep 5

SQLcl: Release 18.4 Production on Wed Apr 17 09:32:10 2019

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

09:32:10 SQL2> -- -------- start session 2 -------- --
09:32:10 SQL2> host sleep 1

09:32:11 SQL2> connect scott/tiger@//localhost/PDB1
Connected.
09:32:11 SQL2> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
4478


09:32:12 SQL2> variable val number
09:32:12 SQL2> variable seq number;
09:32:12 SQL2> exec :seq:=5; :val:=1;

PL/SQL procedure successfully completed.

09:32:12 SQL2> /* TM lock solution */
09:32:12 SQL2>  lock table T in share mode;

09:32:14 SQL1> select session_id,lock_type,mode_held,mode_requested,lock_id1,lock_id2,blocking_others from dba_locks where lock_type in ('DML','Transaction','PL/SQL User Lock');
  SESSION_ID LOCK_TYPE     MODE_HELD    MODE_REQUESTED   LOCK_ID1   LOCK_ID2   BLOCKING_OTHERS
        4478 DML           None         Share            73192      0          Not Blocking
        4479 DML           Row-X (SX)   None             73192      0          Blocking
        4479 Transaction   Exclusive    None             655386     430384     Not Blocking


09:32:14 SQL1> -- -------- continue session 1 while session 2 waits
09:32:14 SQL1> exec :seq:=1; :val:=3;

PL/SQL procedure successfully completed.

09:32:17 SQL1> UPDATE T SET VAL2 = :val WHERE SEQ = :seq;

1 row updated.

09:32:17 SQL1> host sleep 1

09:32:18 SQL1> commit;

Lock succeeded.


Commit complete.

09:32:18 SQL2> UPDATE T SET VAL1 = :val WHERE SEQ < :seq;
09:32:18 SQL1> select * from T;

4 rows updated.

09:32:18 SQL2> commit;

Commit complete.

09:32:18 SQL2> select * from T;
  SEQ VAL1   VAL2
    1 1      3
    2 1      0
    3 1      0
    4 1      2
    5 0      0


09:32:18 SQL1> -- -------- end session 1

  SEQ VAL1   VAL2
    1 1      3
    2 1      0
    3 1      0
    4 1      2
    5 0      0


09:32:18 SQL2> -- -------- end session 2

09:32:18 SQL2>
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.2.0.0.0

同样的例子没有共享锁,我们看到事务排他锁(当更新遇到被另一个事务锁定的行时的锁)导致死锁:

SQLcl: Release 18.4 Production on Wed Apr 17 09:39:35 2019

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

SQL>
SQL> set echo on time on define off sqlprompt "SQL1> " linesize 69 pagesize 1000
09:39:35 SQL1> set sqlformat ansiconsole
09:39:35 SQL1> connect sys/oracle@//localhost/PDB1 as sysdba
Connected.
09:39:36 SQL1>
09:39:36 SQL1> grant dba to scott identified by tiger;

Grant succeeded.

09:39:36 SQL1> connect scott/tiger@//localhost/PDB1
Connected.
09:39:36 SQL1>
09:39:36 SQL1> exec begin execute immediate 'drop table T'; exception when others then null; end;

PL/SQL procedure successfully completed.

09:39:37 SQL1> CREATE TABLE T (
  2    SEQ NUMBER(10) constraint T_SEQ PRIMARY KEY,
  3    VAL1 VARCHAR2(10),
  4    VAL2 VARCHAR2(10)
  5  );

Table created.

09:39:37 SQL1> insert into T select rownum , 0 , 0 from xmltable('1 to 5');

5 rows created.

09:39:37 SQL1> commit;

Commit complete.

09:39:37 SQL1> -- -------- start session 1
09:39:37 SQL1> connect scott/tiger@//localhost/PDB1
Connected.
09:39:37 SQL1>
09:39:37 SQL1> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
4479


09:39:37 SQL1> variable val number
09:39:37 SQL1> variable seq number;
09:39:37 SQL1> exec :seq:=4; :val:=2;

PL/SQL procedure successfully completed.

09:39:37 SQL1> UPDATE T SET VAL2 = :val WHERE SEQ = :seq;

1 row updated.

09:39:37 SQL1> -- -------- call session 2
09:39:37 SQL1> host sql /nolog @/tmp/sql2.sql < /dev/null & :

09:39:37 SQL1> host sleep 5

SQLcl: Release 18.4 Production on Wed Apr 17 09:39:38 2019

Copyright (c) 1982, 2019, Oracle.  All rights reserved.

09:39:38 SQL2> -- -------- start session 2 -------- --
09:39:38 SQL2> host sleep 1

09:39:39 SQL2> connect scott/tiger@//localhost/PDB1
Connected.
09:39:39 SQL2> select sys_context('userenv','sid') from dual;
SYS_CONTEXT('USERENV','SID')
4478


09:39:40 SQL2> variable val number
09:39:40 SQL2> variable seq number;
09:39:40 SQL2> exec :seq:=5; :val:=1;

PL/SQL procedure successfully completed.

09:39:40 SQL2> /* TM lock solution */
09:39:40 SQL2>  --lock table T in share mode;
09:39:40 SQL2> UPDATE T SET VAL1 = :val WHERE SEQ < :seq;

09:39:42 SQL1> select session_id,lock_type,mode_held,mode_requested,lock_id1,lock_id2,blocking_others from dba_locks where lock_type in ('DML','Transaction','PL/SQL User Lock');
  SESSION_ID LOCK_TYPE     MODE_HELD    MODE_REQUESTED   LOCK_ID1   LOCK_ID2   BLOCKING_OTHERS
        4478 Transaction   None         Exclusive        655368     430383     Not Blocking
        4479 DML           Row-X (SX)   None             73194      0          Not Blocking
        4478 DML           Row-X (SX)   None             73194      0          Not Blocking
        4479 Transaction   Exclusive    None             655368     430383     Blocking
        4478 Transaction   Exclusive    None             589838     281188     Not Blocking


09:39:46 SQL1> -- -------- continue session 1 while session 2 waits
09:39:46 SQL1> exec :seq:=1; :val:=3;

PL/SQL procedure successfully completed.

09:39:46 SQL1> UPDATE T SET VAL2 = :val WHERE SEQ = :seq;

1 row updated.

09:39:47 SQL1> host sleep 1

UPDATE T SET VAL1 = :val WHERE SEQ < :seq
             *
ERROR at line 1:
ORA-00060: deadlock detected while waiting for resource

09:39:47 SQL2> commit;

Commit complete.

09:39:47 SQL2> select * from T;
  SEQ VAL1   VAL2
    1 0      0
    2 0      0
    3 0      0
    4 0      0
    5 0      0


09:39:47 SQL2> -- -------- end session 2

09:39:47 SQL2>
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.2.0.0.0

09:39:48 SQL1> commit;

Commit complete.

09:39:48 SQL1> select * from T;
  SEQ VAL1   VAL2
    1 0      3
    2 0      0
    3 0      0
    4 0      2
    5 0      0


09:39:48 SQL1> -- -------- end session 1

此共享锁会阻止所有并发修改,甚至是对通过引用完整性链接的 table 的一些修改,因此请注意对它们的整体写入 activity。另一种解决方案是使用带有 dbms_lock 的自定义用户锁来序列化两组更新。

此致, 弗兰克