Spring 使用 Processor 批量过滤记录

Question

I'm working on a Spring Batch job that creates a string that is based off of sql insert, delete, or update statements. It reads in a flatfile where the first three characters of each line are either add, chg, or del.

示例：

ADD123456001SOUTHLAND PAPER INCORPORATED  ... //more info
CHG123456002GUERNSEY BIG DEAL FAIRFAX     ...//more info
DEL123456002GUERNSEY BIG DEAL FAIRFAX     ...//more info

From the above statements my ItemReader will generate three strings: insert into ..., update ... and delete .... The reader reads in the entire flatfile, returns an arraylist of these strings to my writer, and my writer will take these strings and write to my database.

Here's my problem. What happens if there's a chg requested before an add is requested? What if I try changing something that's already deleted?

I read up on ItemProcessor on SpringDocs and the description of filtering processes is exactly what I'm trying to do:

For example, consider a batch job that reads a file containing three different types of records: records to insert, records to update, and records to delete. If record deletion is not supported by the system, then we would not want to send any "delete" records to the ItemWriter. But, since these records are not actually bad records, we would want to filter them out, rather than skip. As a result, the ItemWriter would receive only "insert" and "update" records.

But the examples of ItemProcessor listed on the docs don't really make sense to me. Can someone make sense of the process to me? Or show me some examples of good ItemProcessing?

Edit: the 6 characters following the command are the id associated in the SQL database.

Answer 1

在问题中描述的情况下，您没有过滤掉记录，您只想更改它们进入的顺序。您最好在较早的步骤中对文件进行排序（以完成您的首先插入，然后是更新，然后是删除）。 ItemProcessor 更适合过滤掉偶尔出现的错误或不相关的输入行。

您可以使用 ItemProcessor 来验证更新或删除的行是否存在，或者要添加的行是否不存在。在这里，我想知道您必须在 ItemProcessor 中执行的查询量（输入文件中每行一个查询）对于可能只偶尔发生的情况来说不会有很多开销。您的选择介于

使用 ItemProcessor 进行过滤（预先对每一行进行查询），或
不执行任何前期查询，但如果 RI 被违反（回滚块并一次重试一行），则让 ItemWriter 跳过这些行，参见 Spring Batch skip exception for ItemWriter。

Spring 使用 Processor 批量过滤记录

Spring Batch filtering out records with Processor

java

spring

spring-batch