MariaDb InnoDB 在进行多次插入时死锁
MariaDb InnoDB deadlock while doing many inserts
如果 MariaDB (10.0.27) 承受压力,我会遇到死锁问题。数据库模式基本上是一个层次结构,叶节点导致:
------------------------
LATEST DETECTED DEADLOCK
------------------------
2020-11-11 15:34:46 0x7fccf00e5700
*** (1) TRANSACTION:
TRANSACTION 2780, ACTIVE 27 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 211 lock struct(s), heap size 24784, 1373 row lock(s), undo log entries 1389
MySQL thread id 383, OS thread handle 140518385858304, query id 614348 172.26.0.1 mydatabase Update
insert into BARCODE (BARCODE_REC_ID, CODE, COLOR_VARIANT_ID, CREATED, EXTERNAL_RECEIPT_NUM, MODIFIED, SEASON_CODE, SEASON_CODE_EB, SEASON_DESCRIPTION, SEASON_YEAR_EB, TYPE, ID) values (5645669455, '021745228', '9404b25d87630677f68d88417ed3efc7', '2018-05-16 16:53:14', '17', '2018-05-16 16:53:14', 'HW18', null, 'Herbst/Winter 2018', 2018, 'VARIANT_RECEIPT_NUM', '5ff302d48259d09c2030e8bdc21749b8')
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 41 page no 15 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2780 lock mode S waiting
Record lock, heap no 340 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 8; hex 8000000150825f21; asc P _!;;
1: len 1; hex 05; asc ;;
2: len 30; hex 623939386139366133316265616632376164396539613463363966643831; asc b998a96a31beaf27ad9e9a4c69fd81; (total 32 bytes);
*** (2) TRANSACTION:
TRANSACTION 2775, ACTIVE 24 sec inserting
mysql tables in use 1, locked 1
247 lock struct(s), heap size 41168, 1915 row lock(s), undo log entries 3256
MySQL thread id 391, OS thread handle 140518177527552, query id 617936 172.26.0.1 mydatabase Update
insert into BARCODE (BARCODE_REC_ID, CODE, COLOR_VARIANT_ID, CREATED, EXTERNAL_RECEIPT_NUM, MODIFIED, SEASON_CODE, SEASON_CODE_EB, SEASON_DESCRIPTION, SEASON_YEAR_EB, TYPE, ID) values (5647403803, '021631613', '053ba855feea779a8e7cbbdaa63e681b', '2019-08-08 10:07:25', '51', '2019-08-08 10:07:25', 'HW19', null, 'Herbst/Winter 2019', 2019, 'VARIANT_RECEIPT_NUM', 'dc3ce352fb06609dc9b4a1ab87d872d1')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 41 page no 15 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2775 lock_mode X locks rec but not gap
Record lock, heap no 340 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 8; hex 8000000150825f21; asc P _!;;
1: len 1; hex 05; asc ;;
2: len 30; hex 623939386139366133316265616632376164396539613463363966643831; asc b998a96a31beaf27ad9e9a4c69fd81; (total 32 bytes);
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 41 page no 36 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2775 lock mode S waiting
Record lock, heap no 208 PHYSICAL RECORD: n_fields 3; compact format; info bits 32
0: len 8; hex 80000001509d484a; asc P HJ;;
1: len 1; hex 05; asc ;;
2: len 30; hex 343638653564396161373030333537623666376531356666623732613735; asc 468e5d9aa700357b6f7e15ffb72a75; (total 32 bytes);
*** WE ROLL BACK TRANSACTION (1)
这是我的叶子 table 的样子:
CREATE TABLE BARCODE (
ID CHAR(32) NOT NULL,
COLOR_VARIANT_ID CHAR(32) NOT NULL,
BARCODE_REC_ID BIGINT NOT NULL,
TYPE ENUM('UPCA', 'UPCE', 'EAN13', 'EAN8', 'VARIANT_RECEIPT_NUM') NOT NULL,
CODE VARCHAR(17) NOT NULL,
CREATED TIMESTAMP NOT NULL,
MODIFIED TIMESTAMP NOT NULL,
EXTERNAL_RECEIPT_NUM INT NOT NULL,
SEASON_CODE VARCHAR(10) NULL,
SEASON_DESCRIPTION VARCHAR(60) NULL,
SEASON_CODE_EB ENUM('CODE_1', 'CODE_5', 'CODE_0') NULL,
SEASON_YEAR_EB INT NULL,
PRIMARY KEY PRI_BARCODE (ID),
INDEX COLOR_VAR_ID (COLOR_VARIANT_ID),
UNIQUE KEY UNI_BARCODE (BARCODE_REC_ID, TYPE)
)
我能够使用使系统处于压力下的输入集重现死锁。有了这个,我一直在玩,一旦我删除了唯一索引 UNI_BARCODE
就没有死锁了。此外,我的数据不会以某种方式交叉,因为我们使用 ActiveMQ 的消费者组来强制对相同产品进行按顺序更新处理。很明显,唯一索引是唯一导致问题的重叠部分。
我花了一些时间试图解决这个问题,但没能成功 :( 如果有任何想法,我将不胜感激 :) 提前致谢!
是的,我在工作场所经常注意到这一点。
同时具有 PRIMARY KEY
和辅助 UNIQUE KEY
的 InnoDB table 很有可能导致死锁。此错误报告中清楚地描述了这种情况的示例:https://bugs.mysql.com/bug.php?id=86812
InnoDB 行级锁定中似乎存在竞争条件。我猜唯一索引上的锁和聚簇索引(主键)上的锁不是原子获取的。
这会影响 MySQL 和 MariaDB,因为它们都使用 InnoDB。它影响了很多版本的 InnoDB,所以升级也无济于事。它会影响 READ-COMMITTED 和 REPEATABLE-READ 事务隔离级别,因此更改它无济于事。
我建议我工作的开发人员有以下三种选择:
重试由于死锁而回滚的事务。这是针对其他类型死锁的通常建议。
悲观锁。在尝试插入之前使用 LOCK TABLES BARCODE WRITE;
。当然 UNLOCK TABLES;
紧随其后,以允许并发会话轮到他们。
重新设计 table,使其不再同时具有主键和唯一键。例如,删除 id
列,使 (BARCODE_REC_ID, TYPE )
上的唯一键成为聚簇索引。
如果 MariaDB (10.0.27) 承受压力,我会遇到死锁问题。数据库模式基本上是一个层次结构,叶节点导致:
------------------------
LATEST DETECTED DEADLOCK
------------------------
2020-11-11 15:34:46 0x7fccf00e5700
*** (1) TRANSACTION:
TRANSACTION 2780, ACTIVE 27 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 211 lock struct(s), heap size 24784, 1373 row lock(s), undo log entries 1389
MySQL thread id 383, OS thread handle 140518385858304, query id 614348 172.26.0.1 mydatabase Update
insert into BARCODE (BARCODE_REC_ID, CODE, COLOR_VARIANT_ID, CREATED, EXTERNAL_RECEIPT_NUM, MODIFIED, SEASON_CODE, SEASON_CODE_EB, SEASON_DESCRIPTION, SEASON_YEAR_EB, TYPE, ID) values (5645669455, '021745228', '9404b25d87630677f68d88417ed3efc7', '2018-05-16 16:53:14', '17', '2018-05-16 16:53:14', 'HW18', null, 'Herbst/Winter 2018', 2018, 'VARIANT_RECEIPT_NUM', '5ff302d48259d09c2030e8bdc21749b8')
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 41 page no 15 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2780 lock mode S waiting
Record lock, heap no 340 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 8; hex 8000000150825f21; asc P _!;;
1: len 1; hex 05; asc ;;
2: len 30; hex 623939386139366133316265616632376164396539613463363966643831; asc b998a96a31beaf27ad9e9a4c69fd81; (total 32 bytes);
*** (2) TRANSACTION:
TRANSACTION 2775, ACTIVE 24 sec inserting
mysql tables in use 1, locked 1
247 lock struct(s), heap size 41168, 1915 row lock(s), undo log entries 3256
MySQL thread id 391, OS thread handle 140518177527552, query id 617936 172.26.0.1 mydatabase Update
insert into BARCODE (BARCODE_REC_ID, CODE, COLOR_VARIANT_ID, CREATED, EXTERNAL_RECEIPT_NUM, MODIFIED, SEASON_CODE, SEASON_CODE_EB, SEASON_DESCRIPTION, SEASON_YEAR_EB, TYPE, ID) values (5647403803, '021631613', '053ba855feea779a8e7cbbdaa63e681b', '2019-08-08 10:07:25', '51', '2019-08-08 10:07:25', 'HW19', null, 'Herbst/Winter 2019', 2019, 'VARIANT_RECEIPT_NUM', 'dc3ce352fb06609dc9b4a1ab87d872d1')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 41 page no 15 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2775 lock_mode X locks rec but not gap
Record lock, heap no 340 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
0: len 8; hex 8000000150825f21; asc P _!;;
1: len 1; hex 05; asc ;;
2: len 30; hex 623939386139366133316265616632376164396539613463363966643831; asc b998a96a31beaf27ad9e9a4c69fd81; (total 32 bytes);
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 41 page no 36 n bits 416 index UNI_BARCODE of table `mydatabase`.`BARCODE` trx id 2775 lock mode S waiting
Record lock, heap no 208 PHYSICAL RECORD: n_fields 3; compact format; info bits 32
0: len 8; hex 80000001509d484a; asc P HJ;;
1: len 1; hex 05; asc ;;
2: len 30; hex 343638653564396161373030333537623666376531356666623732613735; asc 468e5d9aa700357b6f7e15ffb72a75; (total 32 bytes);
*** WE ROLL BACK TRANSACTION (1)
这是我的叶子 table 的样子:
CREATE TABLE BARCODE (
ID CHAR(32) NOT NULL,
COLOR_VARIANT_ID CHAR(32) NOT NULL,
BARCODE_REC_ID BIGINT NOT NULL,
TYPE ENUM('UPCA', 'UPCE', 'EAN13', 'EAN8', 'VARIANT_RECEIPT_NUM') NOT NULL,
CODE VARCHAR(17) NOT NULL,
CREATED TIMESTAMP NOT NULL,
MODIFIED TIMESTAMP NOT NULL,
EXTERNAL_RECEIPT_NUM INT NOT NULL,
SEASON_CODE VARCHAR(10) NULL,
SEASON_DESCRIPTION VARCHAR(60) NULL,
SEASON_CODE_EB ENUM('CODE_1', 'CODE_5', 'CODE_0') NULL,
SEASON_YEAR_EB INT NULL,
PRIMARY KEY PRI_BARCODE (ID),
INDEX COLOR_VAR_ID (COLOR_VARIANT_ID),
UNIQUE KEY UNI_BARCODE (BARCODE_REC_ID, TYPE)
)
我能够使用使系统处于压力下的输入集重现死锁。有了这个,我一直在玩,一旦我删除了唯一索引 UNI_BARCODE
就没有死锁了。此外,我的数据不会以某种方式交叉,因为我们使用 ActiveMQ 的消费者组来强制对相同产品进行按顺序更新处理。很明显,唯一索引是唯一导致问题的重叠部分。
我花了一些时间试图解决这个问题,但没能成功 :( 如果有任何想法,我将不胜感激 :) 提前致谢!
是的,我在工作场所经常注意到这一点。
同时具有 PRIMARY KEY
和辅助 UNIQUE KEY
的 InnoDB table 很有可能导致死锁。此错误报告中清楚地描述了这种情况的示例:https://bugs.mysql.com/bug.php?id=86812
InnoDB 行级锁定中似乎存在竞争条件。我猜唯一索引上的锁和聚簇索引(主键)上的锁不是原子获取的。
这会影响 MySQL 和 MariaDB,因为它们都使用 InnoDB。它影响了很多版本的 InnoDB,所以升级也无济于事。它会影响 READ-COMMITTED 和 REPEATABLE-READ 事务隔离级别,因此更改它无济于事。
我建议我工作的开发人员有以下三种选择:
重试由于死锁而回滚的事务。这是针对其他类型死锁的通常建议。
悲观锁。在尝试插入之前使用
LOCK TABLES BARCODE WRITE;
。当然UNLOCK TABLES;
紧随其后,以允许并发会话轮到他们。重新设计 table,使其不再同时具有主键和唯一键。例如,删除
id
列,使(BARCODE_REC_ID, TYPE )
上的唯一键成为聚簇索引。