Java MySQL 批量插入与正常

Question

我试图使插入到 mysql 的速度更快，所以我写了一个带有批量插入的代码。问题是，当我运行以下代码时，两种方法的 结果相同 （经过的时间：2 分钟或 173 秒）。欢迎对此事发表任何评论。

MySQLConn mysqlconn = new MySQLConn(db,username,pass);
    Connection conn =mysqlconn.getConnection();
    PreparedStatement ps = null;

    String query = "insert into table (column) values (?)";

    System.out.println("bulk insert started with "+10000 );
    long startTime = System.currentTimeMillis();

    try {
        ps = conn.prepareStatement(query);
        long start = System.currentTimeMillis();
        for(int i =0; i<10000;i++){

            ps.setString(1, "Name"+i);

            ps.addBatch();

            if(i%1000 == 0) ps.executeBatch();
        }
        ps.executeBatch();

        System.out.println("Time Taken="+(System.currentTimeMillis()-start));

    } catch (SQLException e) {
        e.printStackTrace();
    }

    System.out.println("End inserting data...");

    long stopTime = System.currentTimeMillis();
    long elapsedTime = (stopTime - startTime); 

    long days = TimeUnit.MILLISECONDS.toDays(elapsedTime);
    long hours = TimeUnit.MILLISECONDS.toHours(elapsedTime);
    long minutes = TimeUnit.MILLISECONDS.toMinutes(elapsedTime);
    long seconds = TimeUnit.MILLISECONDS.toSeconds(elapsedTime);


    System.out.println("Time elapsed: "+minutes+"min, or "+seconds+"sec");



    System.out.println("normal insert started with "+10000 );

    long startTimeNormal = System.currentTimeMillis();

    for(int i=0;i<10000;i++){ 
        String ins="Data"+i;
        Insert insert = new Insert(conn,ins,"db.table", "column" );
    }
    System.out.println("End inserting data...");

    long stopTimeNormal = System.currentTimeMillis();
    long elapsedTimeNormal = (stopTimeNormal - startTimeNormal); 

    long daysN = TimeUnit.MILLISECONDS.toDays(elapsedTime);
    long hoursN = TimeUnit.MILLISECONDS.toHours(elapsedTime);
    long minutesN = TimeUnit.MILLISECONDS.toMinutes(elapsedTime);
    long secondsN = TimeUnit.MILLISECONDS.toSeconds(elapsedTime);


    System.out.println("Time elapsed: "+minutesN+"min, or "+secondsN+"sec");

Answer 1

您使用 addBatch 和 executeBatch，这很好，但您还需要将连接上的自动提交设置为 false 以实现更快的执行时间。

大致情况：

Connection conn = mysqlconn.getConnection();
conn.setAutoCommit(false); // here

PreparedStatement ps = null;

String query = "insert into table (column) values (?)";

System.out.println("bulk insert started with "+10000 );
long startTime = System.currentTimeMillis();

try {
    ps = conn.prepareStatement(query);
    long start = System.currentTimeMillis();
    for(int i =0; i<10000;i++){

        ps.setString(1, "Name"+i);

        ps.addBatch();

        if(i%1000 == 0) {
            ps.executeBatch();
            conn.commit(); // here
        }
    }
    ps.executeBatch();
    conn.commit(); // here

    System.out.println("Time Taken="+(System.currentTimeMillis()-start));

} catch (SQLException e) {
    e.printStackTrace();
}

Answer 2

您需要将自动提交设置为 false

conn.setAutoCommit(false);

并在每次 executeBath()

之后手动执行提交

example

Java MySQL 批量插入与正常

Java MySQL bulk insert vs normal

java

mysql

insert

bulk