多个 Spring 批处理作业并发执行导致死锁

Question

我正在使用 spring batch 4.0.0，spring boot 2.2.0，java jdk 12.0.2，db sql 服务器2016.

多个 Spring 批处理作业并发执行导致 Spring 批处理元数据表中出现死锁。

我尝试了不同的解决方案：

我在元数据表上设置身份

ALTER TABLE [OWN].[BATCH_JOB_EXECUTION_SEQ]
ADD CONSTRAINT [PK_BATCH_JOB_EXECUTION_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

ALTER TABLE [OWN].[BATCH_JOB_SEQ]
ADD CONSTRAINT [PK_BATCH_JOB_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

ALTER TABLE [OWN].[BATCH_STEP_EXECUTION_SEQ]
ADD CONSTRAINT [PK_BATCH_STEP_EXECUTION_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

我确实覆盖了 JobRepository 和 JobExplorer 来配置隔离级别

    @Override
    protected JobRepository createJobRepository() throws Exception {
        JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
        factory.setDatabaseType(DATABASE_TYPE);
        factory.setIsolationLevelForCreate(ISOLATION_REPEATABLE_READ);
        factory.setDataSource(dataSource);
        factory.setTransactionManager(getTransactionManager());
        factory.setTablePrefix(TABLE_PREFIX);
        factory.afterPropertiesSet();
        return factory.getObject();
    }

    @Override
    protected JobExplorer createJobExplorer() throws Exception {
        JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
        jobExplorerFactoryBean.setDataSource(dataSource);
        jobExplorerFactoryBean.afterPropertiesSet();
        jobExplorerFactoryBean.setTablePrefix(TABLE_PREFIX);
        return jobExplorerFactoryBean.getObject();
    }

但是当我开始多项工作时，我得到 often/always 相同的错误：

Could not increment identity; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Transaction was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Answer 1

我已经找到解决方案并将其发布给有同样问题的任何人；我从来没有找到一个完整而全面的解决方案，所以我想解释一下如何解决它以及它为什么会发生，让我们开始吧。这个死锁是由BATCH_STEP_EXECUTION_SEQ、BATCH_JOB_EXECUTION_SEQ、BATCH_JOB_SEQ表的update和delete引起的，但为什么会这样呢？也许是因为事情没有得到妥善管理，但真正的问题是 sql 服务器开始使用来自 sql 服务器 2012 的序列，所以 spring 批处理不得不使用这个表来管理序列，我对于那些使用比 2012 年更旧版本的 sql 服务器的人没有解决方案，本指南仅适用于 2012.

的版本

1.在您的数据库中创建序列。

注意从尚未处理的 ID 开始序列，否则如果您从 1 开始，则必须清理使用作业 ID 的 SPRING 表和可能的表。

DROP TABLE [OWN].[BATCH_JOB_SEQ]
GO


DROP TABLE [OWN].[BATCH_JOB_EXECUTION_SEQ]
GO


DROP TABLE [OWN].[BATCH_STEP_EXECUTION_SEQ]
GO


CREATE SEQUENCE [OWN].[BATCH_JOB_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO


CREATE SEQUENCE [OWN].[BATCH_JOB_EXECUTION_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO

CREATE SEQUENCE [OWN].[BATCH_STEP_EXECUTION_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO

2。也覆盖 JobRepository 方法和 JobExplorer（我个人必须添加它，否则我会遇到有关 spring 批处理表前缀的问题）。

将隔离级别设置为 REPEATABLE_READ 因为否则你将在 DB 上出现新的并发错误（个人没有这个设置我在 BATCH_JOB_EXECUTION 或 BATCH_JOB_INSTANCE i 上遇到问题刚才不记得了）。

@Component
public class SQLServerConfig extends DefaultBatchConfigurer{

  @Autowired
  private DataSource dataSource;

  @Value("${spring.batch.tablePrefix}")
  private String tablePrefix;

  @Override
  protected JobRepository createJobRepository() throws Exception {
      JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
      factory.setDatabaseType(DatabaseType.SQLSERVER.name());
      factory.setDataSource(dataSource);
      factory.setTransactionManager(getTransactionManager());
      factory.setTablePrefix(tablePrefix);
      factory.setIncrementerFactory(new CustomDataFieldMaxValueIncrementerFactory(dataSource));
      factory.setIsolationLevelForCreate("ISOLATION_REPEATABLE_READ");
      factory.afterPropertiesSet();
      return factory.getObject();
  }

  @Override
  protected JobExplorer createJobExplorer() throws Exception {
      JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
      jobExplorerFactoryBean.setDataSource(dataSource);
      jobExplorerFactoryBean.afterPropertiesSet();
      jobExplorerFactoryBean.setTablePrefix(tablePrefix);
      return jobExplorerFactoryBean.getObject();
  }

}

3。设置 CustomDataFieldIncrementerFactory 因为现在它开始从中管理序列。

 public class CustomDataFieldMaxValueIncrementerFactory extends DefaultDataFieldMaxValueIncrementerFactory {


  private DataSource dataSource;


  public CustomDataFieldMaxValueIncrementerFactory(DataSource dataSource) {
      super(dataSource);
      this.dataSource = dataSource;
  }


  @Override
  public DataFieldMaxValueIncrementer getIncrementer(String incrementerType, String incrementerName) {
      DatabaseType databaseType = DatabaseType.valueOf(incrementerType.toUpperCase());
      DataFieldMaxValueIncrementer dataFieldMaxValueIncrementer = null;
      if (databaseType == DatabaseType.SQLSERVER) {
          dataFieldMaxValueIncrementer =  new CustomSqlServerMaxValueIncrementer(dataSource, incrementerName);
      } else {
          dataFieldMaxValueIncrementer = super.getIncrementer(incrementerType, incrementerName);
      }
      return dataFieldMaxValueIncrementer;
  }
}

就是这样，现在运行良好！！！

希望对大家有用。

多个 Spring 批处理作业并发执行导致死锁

Multiple Spring Batch jobs executing concurrently causing deadlocks

java

sql-server

spring

spring-batch

spring-boot