如何处理固定长度格式的数据？
How can I handle fixedLengthformat data?
我处理一些固定长度的数据。
所以我使用 bindy 组件来处理这些数据。
该文件只有一条记录。
记录有页眉、多主体和页脚。
header record (total length : 20)
1 : record_type (length : 1)
VOLTE : service_type (length : 5)
20190515 : creation date (length : 8)
3 : custom_flag (length : 6)
3 body records (total length : 20)
2 : record_type (length : 1)
01012345678 : mobile number (length : 11)
20190515 : call start date (length : 8)
footer records (total length : 20)
3 : record_type (length : 1)
AAAA.DAT : FILE NAME (length : 19)
真实数据
1VOLTE20190515 32010123456782019051520101234567820190516201012345678201905173AAAA.DAT
我定义了如下数据格式。
页眉
@FixedLengthRecord(length=20, paddingChar=' ')
public class VoLTEHeader {
@DataField(pos=1, length=1, trim=true)
String record_type;
@DataField(pos=2, length=5, trim=true)
String service_type;
@DataField(pos=7, length=8, trim=true)
String creation_date;
@DataField(pos=15, length=6, trim=true, align="R")
String custom_flag;
页脚
@FixedLengthRecord(length=20, paddingChar=' ')
public class VoLTEFooter {
@DataField(pos=1, length=1, trim=true)
String record_type;
@DataField(pos=2, length=19, trim=true)
String file_name;
正文
@FixedLengthRecord(length=20, header=VoLTEHeader.class, footer=VoLTEFooter.class)
public class VoLTEBody implements Serializable {
@DataField(pos=1, length=1,trim=true)
String record_type;
@DataField(pos=2, length=11,trim=true)
String mobile_number;
@DataField(pos=13, length=8,trim=true)
String call_start_date;
我执行了骆驼路线，但出现异常
java.lang.IllegalArgumentException: Size of the record: 100 is not equal to the value provided in the model: 20
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.createModel(BindyFixedLengthDataFormat.java:295) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.unmarshal(BindyFixedLengthDataFormat.java:209) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:69) ~[camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.23.2.jar:2.23.2]
我认为 fixedLengthDataFormat 不一定需要在多行中创建。
我该如何解决这个问题？
Camel bindy 将 header 定义为“文件/流开头的单个 header 记录”，并将页脚定义为“文件/流末尾的单个页脚记录"，请参阅[camel-bindy 文档][1]。
您的测试数据在同一行的 multi-payload 部分前后有元数据，您不能使用 Bindy header/footer 模型来解析它。
而是为 header、页脚和一个 body 创建独立的 Bindy 模型（基本上从 VoLTEBody 中删除 "header=VoLTEHeader.class, footer=VoLTEFooter.class"）并单独处理 header、页脚和正文：
public class MyRouteBuilderTest extends CamelTestSupport {
@Produce(uri="direct:start")
private ProducerTemplate producer;
@Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
public void configure() {
DataFormat bindyHeader = new BindyFixedLengthDataFormat(VoLTEHeader.class);
DataFormat bindyFooter = new BindyFixedLengthDataFormat(VoLTEFooter.class);
DataFormat bindyOneBody = new BindyFixedLengthDataFormat(VoLTEBody.class);
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
// get the VoLTE header string into the Camel-body and unmarshal, then put VoLTE header object into Camel-header for later use
.setBody().groovy("request.body.substring(0,20)")
.unmarshal(bindyHeader)
.setHeader("header", body())
// restore VoLTE record string to Camel-body, get the VoLTE footer string into the Camel-body and unmarshal, then put footer VoLTE object into Camel-header for later use
.setBody().header("record")
.setBody().groovy("request.body.substring(request.body.length()-20,request.body.length())")
.unmarshal(bindyFooter)
.setHeader("footer", body())
// restore VoLTE record string to Camel-body, get the multi-bodies string into the Camel-body
.setBody().header("record")
.setBody().groovy("request.body.substring(20,request.body.length()-20)")
// Split VoLTE bodies string to each 20 char length, unmarshal each and finally put the list of VoLTE body objects into a Camel-header
.setBody().groovy("request.body.split('(?<=\\G.{20})')")
.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
.end()
// now do something with the unmarshalled objects in Camel-headers "header" (type VoLTEHeader), "footer" (type VoLTEFooter) and "bodyList" (type List<VoLTEBody>)
.log("VoLTEHeader: ${header.header}")
.log("VoLTEBody*: ${header.bodyList}")
.log("VoLTEFooter: ${header.footer}")
;
}
};
}
@Test
public void test() throws Exception {
producer.sendBody("1VOLTE20190515 32010123456782019051520101234567820190516201012345678201905173AAAA.DAT ");
}
}
@FixedLengthRecord(length = 20, paddingChar = ' ')
public class VoLTEHeader {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 5, trim = true)
String service_type;
@DataField(pos = 7, length = 8, trim = true)
String creation_date;
@DataField(pos = 15, length = 6, trim = true, align = "R")
String custom_flag;
@Override
public String toString() {
return String.format("VoLTEHeader[record_type=%s, service_type=%s, creation_date=%s, custom_flag=%s]", record_type, service_type, creation_date, custom_flag);
}
}
@FixedLengthRecord(length = 20)
public class VoLTEBody {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 11, trim = true)
String mobile_number;
@DataField(pos = 13, length = 8, trim = true)
String call_start_date;
@Override
public String toString() {
return String.format("VoLTEBody[record_type=%s, mobile_number=%s, call_start_date=%s]", record_type, mobile_number, call_start_date);
}
}
@FixedLengthRecord(length = 20, paddingChar = ' ')
public class VoLTEFooter {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 19, trim = true)
String file_name;
@Override
public String toString() {
return String.format("VoLTEFooter[record_type=%s, file_name=%s]", record_type, file_name);
}
}
输出：
[main] INFO route1 - VoLTEHeader: VoLTEHeader[record_type=1, service_type=VOLTE, creation_date=20190515, custom_flag=3]
[main] INFO route1 - VoLTEBody*: [VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190515], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190516], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]]
[main] INFO route1 - VoLTEFooter: VoLTEFooter[record_type=3, file_name=AAAA.DAT ]
在路线的尽头，您应该在 Camel-header "header" 中有一个类型为 VoLTEHeader 的 object，在 [=33= 中有一个类型为 VoLTEFooter 的 object ] "footer" 和 Camel-header "bodyList" 中的 VoLTEBody 列表。您现在可以使用这些进行处理。
谢谢。我通过您的指导实施了它。
我认为 Bindy 组件支持页眉、页脚和正文。
所以，我考虑了以下逻辑。
传入文件 -> 拆分（20 长度） -> 聚合拆分文件 -> 解组
我使用包含页眉、页脚和正文的聚合文件。
下面是代码。
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
.setBody().groovy("request.body.split('(?<=\\G.{20})')")
.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
.end()
.log("VoLTEBody*: ${header.bodyList}")
...
VoLTEBody
@FixedLengthRecord(length=20, header=VoLTEHeader.class, footer=VoLTETailer.class)
public class VoLTEBody {
@DataField(pos=1, length=1,trim=true)
String record_type;
但是出现如下错误
Stacktrace
------------------------------------------------------------------------------
---------------------------------------------------------
java.lang.IllegalArgumentException: No records have been defined in the file
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.unmarshal(BindyFixedLengthDataFormat.java:250) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:69) ~[camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
我认为如果在 VoLTEBody 中正确定义了关于页眉和页脚的注释，这个过程就没问题 class。
我该如何处理这个问题？
此外，我执行了另一个测试。
在这种情况下，没有分裂。
传入文件 -> 解组 (bindyOneBody)
路由器
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
//.setBody().groovy("request.body.split('(?<=\\G.{20})')")
//.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
结果如下。
在这个结果中，我们找不到 VoLTEHeader 和 VoLTEFooter 结构。这很好。
2019-05-16 15:22:15,798 DEBUG BindyFixedLengthDataFormat - Graph of objects created: {camel.dataformat.volte.VoLTEBody_sample=VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]}
2019-05-16 15:22:15,798 INFO route1 - [VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190515], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190516], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]]
2019-05-16 15:22:15,798 INFO MyRouteBuilderTest - ********************************************************************************
我处理一些固定长度的数据。 所以我使用 bindy 组件来处理这些数据。
该文件只有一条记录。 记录有页眉、多主体和页脚。
header record (total length : 20)
1 : record_type (length : 1)
VOLTE : service_type (length : 5)
20190515 : creation date (length : 8)
3 : custom_flag (length : 6)
3 body records (total length : 20)
2 : record_type (length : 1)
01012345678 : mobile number (length : 11)
20190515 : call start date (length : 8)
footer records (total length : 20)
3 : record_type (length : 1)
AAAA.DAT : FILE NAME (length : 19)
真实数据
1VOLTE20190515 32010123456782019051520101234567820190516201012345678201905173AAAA.DAT
我定义了如下数据格式。
页眉
@FixedLengthRecord(length=20, paddingChar=' ')
public class VoLTEHeader {
@DataField(pos=1, length=1, trim=true)
String record_type;
@DataField(pos=2, length=5, trim=true)
String service_type;
@DataField(pos=7, length=8, trim=true)
String creation_date;
@DataField(pos=15, length=6, trim=true, align="R")
String custom_flag;
页脚
@FixedLengthRecord(length=20, paddingChar=' ')
public class VoLTEFooter {
@DataField(pos=1, length=1, trim=true)
String record_type;
@DataField(pos=2, length=19, trim=true)
String file_name;
正文
@FixedLengthRecord(length=20, header=VoLTEHeader.class, footer=VoLTEFooter.class)
public class VoLTEBody implements Serializable {
@DataField(pos=1, length=1,trim=true)
String record_type;
@DataField(pos=2, length=11,trim=true)
String mobile_number;
@DataField(pos=13, length=8,trim=true)
String call_start_date;
我执行了骆驼路线，但出现异常
java.lang.IllegalArgumentException: Size of the record: 100 is not equal to the value provided in the model: 20
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.createModel(BindyFixedLengthDataFormat.java:295) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.unmarshal(BindyFixedLengthDataFormat.java:209) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:69) ~[camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:138) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:101) [camel-core-2.23.2.jar:2.23.2]
我认为 fixedLengthDataFormat 不一定需要在多行中创建。
我该如何解决这个问题？
Camel bindy 将 header 定义为“文件/流开头的单个 header 记录”，并将页脚定义为“文件/流末尾的单个页脚记录"，请参阅[camel-bindy 文档][1]。
您的测试数据在同一行的 multi-payload 部分前后有元数据，您不能使用 Bindy header/footer 模型来解析它。
而是为 header、页脚和一个 body 创建独立的 Bindy 模型（基本上从 VoLTEBody 中删除 "header=VoLTEHeader.class, footer=VoLTEFooter.class"）并单独处理 header、页脚和正文：
public class MyRouteBuilderTest extends CamelTestSupport {
@Produce(uri="direct:start")
private ProducerTemplate producer;
@Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
public void configure() {
DataFormat bindyHeader = new BindyFixedLengthDataFormat(VoLTEHeader.class);
DataFormat bindyFooter = new BindyFixedLengthDataFormat(VoLTEFooter.class);
DataFormat bindyOneBody = new BindyFixedLengthDataFormat(VoLTEBody.class);
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
// get the VoLTE header string into the Camel-body and unmarshal, then put VoLTE header object into Camel-header for later use
.setBody().groovy("request.body.substring(0,20)")
.unmarshal(bindyHeader)
.setHeader("header", body())
// restore VoLTE record string to Camel-body, get the VoLTE footer string into the Camel-body and unmarshal, then put footer VoLTE object into Camel-header for later use
.setBody().header("record")
.setBody().groovy("request.body.substring(request.body.length()-20,request.body.length())")
.unmarshal(bindyFooter)
.setHeader("footer", body())
// restore VoLTE record string to Camel-body, get the multi-bodies string into the Camel-body
.setBody().header("record")
.setBody().groovy("request.body.substring(20,request.body.length()-20)")
// Split VoLTE bodies string to each 20 char length, unmarshal each and finally put the list of VoLTE body objects into a Camel-header
.setBody().groovy("request.body.split('(?<=\\G.{20})')")
.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
.end()
// now do something with the unmarshalled objects in Camel-headers "header" (type VoLTEHeader), "footer" (type VoLTEFooter) and "bodyList" (type List<VoLTEBody>)
.log("VoLTEHeader: ${header.header}")
.log("VoLTEBody*: ${header.bodyList}")
.log("VoLTEFooter: ${header.footer}")
;
}
};
}
@Test
public void test() throws Exception {
producer.sendBody("1VOLTE20190515 32010123456782019051520101234567820190516201012345678201905173AAAA.DAT ");
}
}
@FixedLengthRecord(length = 20, paddingChar = ' ')
public class VoLTEHeader {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 5, trim = true)
String service_type;
@DataField(pos = 7, length = 8, trim = true)
String creation_date;
@DataField(pos = 15, length = 6, trim = true, align = "R")
String custom_flag;
@Override
public String toString() {
return String.format("VoLTEHeader[record_type=%s, service_type=%s, creation_date=%s, custom_flag=%s]", record_type, service_type, creation_date, custom_flag);
}
}
@FixedLengthRecord(length = 20)
public class VoLTEBody {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 11, trim = true)
String mobile_number;
@DataField(pos = 13, length = 8, trim = true)
String call_start_date;
@Override
public String toString() {
return String.format("VoLTEBody[record_type=%s, mobile_number=%s, call_start_date=%s]", record_type, mobile_number, call_start_date);
}
}
@FixedLengthRecord(length = 20, paddingChar = ' ')
public class VoLTEFooter {
@DataField(pos = 1, length = 1, trim = true)
String record_type;
@DataField(pos = 2, length = 19, trim = true)
String file_name;
@Override
public String toString() {
return String.format("VoLTEFooter[record_type=%s, file_name=%s]", record_type, file_name);
}
}
输出：
[main] INFO route1 - VoLTEHeader: VoLTEHeader[record_type=1, service_type=VOLTE, creation_date=20190515, custom_flag=3]
[main] INFO route1 - VoLTEBody*: [VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190515], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190516], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]]
[main] INFO route1 - VoLTEFooter: VoLTEFooter[record_type=3, file_name=AAAA.DAT ]
在路线的尽头，您应该在 Camel-header "header" 中有一个类型为 VoLTEHeader 的 object，在 [=33= 中有一个类型为 VoLTEFooter 的 object ] "footer" 和 Camel-header "bodyList" 中的 VoLTEBody 列表。您现在可以使用这些进行处理。
谢谢。我通过您的指导实施了它。 我认为 Bindy 组件支持页眉、页脚和正文。
所以，我考虑了以下逻辑。
传入文件 -> 拆分（20 长度） -> 聚合拆分文件 -> 解组
我使用包含页眉、页脚和正文的聚合文件。
下面是代码。
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
.setBody().groovy("request.body.split('(?<=\\G.{20})')")
.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
.end()
.log("VoLTEBody*: ${header.bodyList}")
...
VoLTEBody
@FixedLengthRecord(length=20, header=VoLTEHeader.class, footer=VoLTETailer.class)
public class VoLTEBody {
@DataField(pos=1, length=1,trim=true)
String record_type;
但是出现如下错误
Stacktrace
------------------------------------------------------------------------------
---------------------------------------------------------
java.lang.IllegalArgumentException: No records have been defined in the file
at org.apache.camel.dataformat.bindy.fixed.BindyFixedLengthDataFormat.unmarshal(BindyFixedLengthDataFormat.java:250) ~[camel-bindy-2.23.2.jar:2.23.2]
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:69) ~[camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.23.2.jar:2.23.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.23.2.jar:2.23.2]
我认为如果在 VoLTEBody 中正确定义了关于页眉和页脚的注释，这个过程就没问题 class。
我该如何处理这个问题？
此外，我执行了另一个测试。 在这种情况下，没有分裂。
传入文件 -> 解组 (bindyOneBody)
路由器
from("direct:start")
// assuming the raw record is in the body, we keep a copy in a Camel-header
.setHeader("record", body())
//.setBody().groovy("request.body.split('(?<=\\G.{20})')")
//.split(body(), AggregationStrategies.flexible().storeInHeader("bodyList").accumulateInCollection(ArrayList.class))
.unmarshal(bindyOneBody)
结果如下。 在这个结果中，我们找不到 VoLTEHeader 和 VoLTEFooter 结构。这很好。
2019-05-16 15:22:15,798 DEBUG BindyFixedLengthDataFormat - Graph of objects created: {camel.dataformat.volte.VoLTEBody_sample=VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]}
2019-05-16 15:22:15,798 INFO route1 - [VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190515], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190516], VoLTEBody[record_type=2, mobile_number=01012345678, call_start_date=20190517]]
2019-05-16 15:22:15,798 INFO MyRouteBuilderTest - ********************************************************************************