是否可以从 Java PreparedStatement 逐行插入?
Is it Possible to Insert Row by Row from a Java PreparedStatement?
我有一个应用程序,我正在使用准备好的语句将 1000 行的批次写入 SQL 服务器数据库。现在,如果这些插入中的任何一个失败,我希望能够将有问题的行写入日志文件,这样数据就不会完全丢失。如果可能的话,我更愿意只将失败的一行写入日志,而不是所有 1000 行,所以我的问题是:
如果一行在插入 table 时失败,假设发生了主键冲突,是阻止了整个批次的插入,还是只有失败的一行?如果批处理失败,我是否可以对准备好的语句进行某种循环,一个一个地执行每个查询,这样我就可以找出失败的行,并只记录该行?或者我实现此目的的唯一方法是为每个插入语句保留一个单独的数组,并在批处理碰巧失败时循环遍历它?
更多细节:
Database Type: SQL Server 2008
Connection Library: java.sql
SQL Statement:
Insert into Table(Column1,Column2) Values (Value1, Value2)
Java代码:
PreparedStatement prpStmt = dbConnection.prepareStatement(insertQuery.toString());
for (List lst : listOfValues){
prpStmt.setString(1,lst[0]);
prpStmt.setString(2,lst[1]);
prpStmt.addBatch();
dbCount++;
if (dbCount == DB_COUNT_LIMIT){
try {
prpStmt.executeBatch();
dbConnection.commit();
dbCount = 0;
prpStmt.clearBatch();
} catch (Exception e){
for (PreparedStatement ps : prpStmt.getBatches()){
if (!logToDBIndividually())
logToFile();
}
}
}
}
If one line fails while inserting into the table, say a Primary Key violation occurs, is the whole batch prevented from inserting, or only the one row that failed?
在发生错误的语句之前执行的语句仍然被执行。批处理中后面的语句可能已执行也可能未执行,由驾驶员自行决定。假设您已经关闭了语句连接的自动提交,就像您在使用 executeBatch()
时应该做的那样,并且看起来您已经完成了,是否提交或回滚已成功进行的更改由您自行决定。
无论驱动程序是否继续执行失败的语句,如果 确实 失败,那么您可以通过检查 返回的数组来确定哪些批处理语句已成功执行=12=] 方法的结果 BatchUpdateException
。有关详细信息,请参阅该方法的文档或 Statement.executeBatch()
的文档。
另请注意,您提供的示例代码存在严重缺陷。如果您的某个更新失败,从而引发异常,重要的是提交或回滚事务,并清除您不想在下一次迭代中重新执行的任何批处理语句环形。此外,捕获普通的 Exception
很少是合适的,这里尤其不合适,因为您希望以不同于其他异常的方式处理 BatchUpdateException
。这样的东西可能会更好:
PreparedStatement prpStmt = dbConnection.prepareStatement(insertQuery.toString());
for (List lst : listOfValues){
prpStmt.setString(1,lst[0]);
prpStmt.setString(2,lst[1]);
prpStmt.addBatch();
dbCount++;
if (dbCount >= DB_COUNT_LIMIT) { // should not be >, but no harm in being safe
try {
prpStmt.executeBatch();
dbConnection.commit();
dbCount = 0;
prpStmt.clearBatch();
} catch (BatchUpdateException bue){
int[] updateCounts = bue.getUpdateCounts();
if (updateCounts.length < dbCount) {
/*
* The first updateCounts.length statements (only) were
* executed successfully. The next one failed, and no more
* were attempted.
*/
} else {
/*
* The failed statements can be identified by having
* updateCounts[i] == Statement.EXECUTE_FAILED
*/
}
// Presumably you want to:
dbConnection.commit();
// Maybe you want to:
dbCount = 0;
prpStmt.clearBatch();
// Otherwise you need to do some other kind of cleanup / retry
}
/*
* no need to catch any other exception, including SQLException, in
* this scope, as it's unlikely that the overall bulk insertion can be
* continued after such an exception.
*/
}
}
If the batch fails, is it possible for me to do some sort of loop on the prepared statement, executing each query one by one, and in that way I can figure out the line that failed, and log only that line?
您可以确定失败的语句,如上所示。这将允许您记录失败。但是,您不能删除询问 Statement
对象的当前批次,也不能删除 via clearBatch()
以外的行,因此如果驱动程序恰好是在第一个错误后停止处理批次的类型,然后从中恢复这样的失败可能并不像您希望的那样简单。不过,需要的信息就在那里;使用索引 for
循环而不是增强的 for
循环来遍历列表可能更容易。
Or is the only way for me to achieve this to keep a separate array of each insert statement, and loop through that if the batch happens to fail?
不,这不是唯一的方法,但我可以想象可以干净利落地实施的变体。但是,使用索引 for
循环,您确实可以非常干净地恢复。
我有一个应用程序,我正在使用准备好的语句将 1000 行的批次写入 SQL 服务器数据库。现在,如果这些插入中的任何一个失败,我希望能够将有问题的行写入日志文件,这样数据就不会完全丢失。如果可能的话,我更愿意只将失败的一行写入日志,而不是所有 1000 行,所以我的问题是:
如果一行在插入 table 时失败,假设发生了主键冲突,是阻止了整个批次的插入,还是只有失败的一行?如果批处理失败,我是否可以对准备好的语句进行某种循环,一个一个地执行每个查询,这样我就可以找出失败的行,并只记录该行?或者我实现此目的的唯一方法是为每个插入语句保留一个单独的数组,并在批处理碰巧失败时循环遍历它?
更多细节:
Database Type: SQL Server 2008
Connection Library: java.sql
SQL Statement:
Insert into Table(Column1,Column2) Values (Value1, Value2)
Java代码:
PreparedStatement prpStmt = dbConnection.prepareStatement(insertQuery.toString());
for (List lst : listOfValues){
prpStmt.setString(1,lst[0]);
prpStmt.setString(2,lst[1]);
prpStmt.addBatch();
dbCount++;
if (dbCount == DB_COUNT_LIMIT){
try {
prpStmt.executeBatch();
dbConnection.commit();
dbCount = 0;
prpStmt.clearBatch();
} catch (Exception e){
for (PreparedStatement ps : prpStmt.getBatches()){
if (!logToDBIndividually())
logToFile();
}
}
}
}
If one line fails while inserting into the table, say a Primary Key violation occurs, is the whole batch prevented from inserting, or only the one row that failed?
在发生错误的语句之前执行的语句仍然被执行。批处理中后面的语句可能已执行也可能未执行,由驾驶员自行决定。假设您已经关闭了语句连接的自动提交,就像您在使用 executeBatch()
时应该做的那样,并且看起来您已经完成了,是否提交或回滚已成功进行的更改由您自行决定。
无论驱动程序是否继续执行失败的语句,如果 确实 失败,那么您可以通过检查 返回的数组来确定哪些批处理语句已成功执行=12=] 方法的结果 BatchUpdateException
。有关详细信息,请参阅该方法的文档或 Statement.executeBatch()
的文档。
另请注意,您提供的示例代码存在严重缺陷。如果您的某个更新失败,从而引发异常,重要的是提交或回滚事务,并清除您不想在下一次迭代中重新执行的任何批处理语句环形。此外,捕获普通的 Exception
很少是合适的,这里尤其不合适,因为您希望以不同于其他异常的方式处理 BatchUpdateException
。这样的东西可能会更好:
PreparedStatement prpStmt = dbConnection.prepareStatement(insertQuery.toString());
for (List lst : listOfValues){
prpStmt.setString(1,lst[0]);
prpStmt.setString(2,lst[1]);
prpStmt.addBatch();
dbCount++;
if (dbCount >= DB_COUNT_LIMIT) { // should not be >, but no harm in being safe
try {
prpStmt.executeBatch();
dbConnection.commit();
dbCount = 0;
prpStmt.clearBatch();
} catch (BatchUpdateException bue){
int[] updateCounts = bue.getUpdateCounts();
if (updateCounts.length < dbCount) {
/*
* The first updateCounts.length statements (only) were
* executed successfully. The next one failed, and no more
* were attempted.
*/
} else {
/*
* The failed statements can be identified by having
* updateCounts[i] == Statement.EXECUTE_FAILED
*/
}
// Presumably you want to:
dbConnection.commit();
// Maybe you want to:
dbCount = 0;
prpStmt.clearBatch();
// Otherwise you need to do some other kind of cleanup / retry
}
/*
* no need to catch any other exception, including SQLException, in
* this scope, as it's unlikely that the overall bulk insertion can be
* continued after such an exception.
*/
}
}
If the batch fails, is it possible for me to do some sort of loop on the prepared statement, executing each query one by one, and in that way I can figure out the line that failed, and log only that line?
您可以确定失败的语句,如上所示。这将允许您记录失败。但是,您不能删除询问 Statement
对象的当前批次,也不能删除 via clearBatch()
以外的行,因此如果驱动程序恰好是在第一个错误后停止处理批次的类型,然后从中恢复这样的失败可能并不像您希望的那样简单。不过,需要的信息就在那里;使用索引 for
循环而不是增强的 for
循环来遍历列表可能更容易。
Or is the only way for me to achieve this to keep a separate array of each insert statement, and loop through that if the batch happens to fail?
不,这不是唯一的方法,但我可以想象可以干净利落地实施的变体。但是,使用索引 for
循环,您确实可以非常干净地恢复。