Spring 应用程序：Azure SQL 服务器插入很慢

Question

我有一个 spring 批处理应用程序，它使用 Azure SQL 服务器作为数据库。

我有一个具有 20 个 vCores

的超大规模 Azure SQL 服务器实例

下面是我的table结构

SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[Schema].[DeleteMe]') AND type in (N'U'))
DROP TABLE [Schema].[DeleteMe]
GO

CREATE TABLE [Schema].[DeleteMe](
[ID] [int] NOT NULL,
[LastName] [char](255) NOT NULL,
[FirstName] [char](255) NOT NULL,
[Age] [int] NOT NULL,
[DOJ] [datetime2](0) NOT NULL,
[Role] [varchar](255) NULL
) ON [PRIMARY]
GO

插入 2800 条记录大约需要 10 秒

public class JDBCSample {
    ...

    public static void main(String[] args) {
        ...

        // Open a connection
        try {
            ...

            JdbcTemplate jdbcTemplate = new JdbcTemplate();
            // Direct SQL Server Datasource
            //jdbcTemplate.setDataSource(ds);

            // Hikari Connection Pooling
            jdbcTemplate.setDataSource(dataSource());

            String deleteSQL = "Delete from DeleteMe";
            jdbcTemplate.update(deleteSQL);

            /* Using PreparedStatement*/
            /**/
            Connection conn = dataSource().getConnection();
            conn.setAutoCommit(true);
            PreparedStatement stmt = conn.prepareStatement("insert into DeleteMe VALUES(?, 'vijas', 'asdf', 36, '2020-10-30', 'safd')");

            try
            {
                for (int id = 1; id <= 2800; id++) {
                    stmt.setInt(1, id);
                    stmt.addBatch();
                }

                stmt.executeBatch();
            }
            catch (SQLException e) {
                System.out.println("Error message: " + e.getMessage());
                return; // Exit if there was an error
            }

            LocalDateTime endTime = LocalDateTime.now();
            System.out.println(dtf.format(endTime));

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Bean(destroyMethod = "close")
    public static DataSource dataSource(){
        HikariConfig hikariConfig = new HikariConfig();
        hikariConfig.setDriverClassName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
        hikariConfig.setJdbcUrl("jdbc:....");
        hikariConfig.setUsername("....");
        hikariConfig.setPassword("....");

        hikariConfig.setMaximumPoolSize(10);
        hikariConfig.setConnectionTestQuery("SELECT 1");
        hikariConfig.setPoolName("springHikariCP");

        /*
        hikariConfig.addDataSourceProperty("dataSource.cachePrepStmts", "true");
        hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSize", "5000");
        hikariConfig.addDataSourceProperty("dataSource.prepStmtCacheSqlLimit", "4096");
        hikariConfig.addDataSourceProperty("dataSource.useServerPrepStmts", "true");

        hikariConfig.addDataSourceProperty("hibernate.jdbc.batch_size", "5000");
        hikariConfig.addDataSourceProperty("hibernate.order_inserts", "true");
        hikariConfig.addDataSourceProperty("hibernate.order_updates", "true");
        hikariConfig.addDataSourceProperty("hibernate.jdbc.batch_versioned_data", "true");
        */

        HikariDataSource dataSource = new HikariDataSource(hikariConfig);

        return dataSource;
    }
}

注意：我已尝试将以下参数添加到 JDBC 属性

statementPoolingCacheSize=10;disableStatementPooling=false;enablePrepareOnFirstPreparedStatementCall=true;sendStringParametersAsUnicode=false;

我想将执行时间进一步减少到不到一秒。我如何实现这一目标？这是生产部署的障碍。

Answer 1

发现自动提交导致延迟 - 将自动提交替换为 connection.commit()

public class JDBCSample {
    ...

    public static void main(String[] args) {
        ...

        // Open a connection
        try {
            ...

            JdbcTemplate jdbcTemplate = new JdbcTemplate();
            // Direct SQL Server Datasource
            //jdbcTemplate.setDataSource(ds);

            // Hikari Connection Pooling
            jdbcTemplate.setDataSource(dataSource());

            String deleteSQL = "Delete from DeleteMe";
            jdbcTemplate.update(deleteSQL);

            /* Using PreparedStatement*/
            /**/
            Connection conn = dataSource().getConnection();
            conn.setAutoCommit(false);
            PreparedStatement stmt = conn.prepareStatement("insert into DeleteMe VALUES(?, 'vijas', 'asdf', 36, '2020-10-30', 'safd')");

            try
            {
                for (int id = 1; id <= 2800; id++) {
                    stmt.setInt(1, id);
                    stmt.addBatch();
                }

                stmt.executeBatch();
                conn.commit();
            }
            catch (SQLException e) {
                System.out.println("Error message: " + e.getMessage());
                return; // Exit if there was an error
            }

            LocalDateTime endTime = LocalDateTime.now();
            System.out.println(dtf.format(endTime));

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Bean(destroyMethod = "close")
    public static DataSource dataSource(){
        HikariConfig hikariConfig = new HikariConfig();
        hikariConfig.setDriverClassName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
        hikariConfig.setJdbcUrl("jdbc:....");
        hikariConfig.setUsername("....");
        hikariConfig.setPassword("....");

        hikariConfig.setMaximumPoolSize(10);
        hikariConfig.setConnectionTestQuery("SELECT 1");
        hikariConfig.setPoolName("springHikariCP");

        HikariDataSource dataSource = new HikariDataSource(hikariConfig);

        return dataSource;
    }
}

Spring 应用程序：Azure SQL 服务器插入很慢

Spring application : Azure SQL Server Insert is slow

java

spring

azure

azure-sql-database

azure-sql-server