放气熵问题
Deflater entropy issue
我正在使用 Deflater
通过网络发送一些消息。
我正在使用:
public static byte[] compress(byte[] data,int level) throws IOException {
Instant starts = Instant.now();
Deflater deflater = new Deflater();
deflater.setInput(data);
deflater.setLevel(/*level*/9);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
deflater.finish();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer); // returns the generated code... index
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
System.out.println("Original: " + data.length / 1024 + " Kb");
System.out.println("Compressed: " + output.length / 1024 + " Kb");
Instant ends = Instant.now();
System.out.println("deflating time:"+Duration.between(starts, ends));
return output;
}
压缩通用字节数组,
和:
msg.setDataString(Base64.getEncoder().encodeToString(data));
将对象嵌入 xml。
有一段时间一切正常,但今天使用我看到的调试器:
msg.dataString
(这是一组几乎没有图像的 rgb 像素(3 字节)数组)
eNrt3e1i4jYQBdCR3/+hu+XTYM1I2IEku+fUbbdRDARMf1wmVxEAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAPCXan/++v/Iltvp6J8Z9bkAAAAAAPArtc1x/ur1z8/r58S8ncTmuP+7v9qKc8+n
3+/n+Rj9GAAAAAAA8HHt/odNrn2KuzP16jVKn7FsjqGojqiOMFcPAAAAAEClSJHLBPo2Ut7J
2x/n1F8N05fXg/bJYzaN7y4O0ngAAAAAAP5hRWBczoJHlU2fvrhEHa1fAvBuJr5c/2vZHO36
z+y4rG5PvX5D/Dk697rc8/beY4rxbHy5CgAAAADAL9dmJth7A9tVWn5thMmmxidmzR9D8Ids
e1mu4fombV9Oi2edzPx84mV5m7cvSyy9KP56W3H6hshn44vPEGJutYzizc0DAAAAAPxUtx1L
k0LycoR9ULBet7QsVRq/TETxS8c1he9rvZh98rjcczZ0P/qIYNSOE3nwPlEcf3stpfEAAAAA
AG9Vpq/5hPUgUp/ctXRYGdPL29uqH2Y7pN725u39JpnJvH0pEvWHfpt95fD1M51/7BFa4wEA
AAAAvkwZuLb8u+JWLJ7kvFHlvIcS9Wq+fXnoY++MkdfJ9mBGvU7Us2L4ien3slU+b6QfpPGD
Lvxhj/5l91phPAAAAADARZWSpuXfqyH1Tt5elsKst/kcjbfvG+ZOB8GrHU8Hafh6R9TOfPty
oFKm7n4vem4Gsfr4CYvRavSn4K9V+tHs0QoAAAAAsJpDz4fYW/ZdrdrXNKodPGdi3qm8vc7i
u6u3vVDTspm8UKYVM+qrID9to0kaY5aJufrBBPvM5rGd1WhHmnvyYvhrGU111QEAAAAA/Ep5
pF6F7ecZ9SRvP082L3FKbV/sfDmumuiu91Ldn7fXvTKrr2Z5fLa6erj9+faHR/98twd+FyCO
Z+0Tv7/Q2Q1X5g4AAAAA/FK3ALSTtterLS0UmdjUNPauRvtuWXJdz54XI+pFkU0b5+1TI+y9
Bpx3mauhyfqCRqevLkFNNQAAAADAh9X55GaLy1Vj+72Mu3ezt9VtGN/enebuy2vjQM5f5+1x
y8NbMTy/7Jv2Hybk1cz+O36T4N0vcH4VRUQrd+mt9/AFAAAAAP51bbDaBhO/STnMbSl6Q+rx
1PKxLf5IK7qPZ+b1yTun32OJj6TIO5rj39iu83lRPpkxlbe3yLbaXUXv2xqj+3p3D9+2Ljx6
upzv7wYAAAAA4O9Qz+UWm5a20Q0OTu6Gm9ns+331SF1MnbdHfnzX5PXuTwGO9Oesb7nzVKzW
OgH0tzxTh+73C57F+ncf0tViw4LLWykpVwIAAAAAfprdLRltbrV7j6th387JRej9mLe/GoxH
Of883/Bd32/1kF7dPfTrYuYqM09z4OTcde/K6DOR7rlvCeSjrBSKGP24R5/ifb83UV110Qbv
hZC9AwAAAMBnHemdbhPHy3fcZufb66R3V+59sGV9ZoZ5x/0eiZ9jVEh+K1rp5N7lapWox/A1
SpL1Ud4e4zb85O4vgXuyl+7h3Lvt+wwoDr2C1d4E6+e5/+6TxgMAAADAV6szt5m8/dXbnL/j
PL1Mht8nAvmsi6bNnbivFWRiSnl3IczOo/7Z00rxzVOfp/H1ZyLFK5h+1nL5O6n+v9z+5fpJ
utKTkvVRu/v8tbFjdbKnqLsUxdVzf0KzR5W9u5scHgAAAAB+jDaceG/5qa/Nz28z2Zmkd8dU
9nSzR5WK93PgfEp5kORHGdXGTKI+0fvT2yD0+aXqvopteJlk10b+mG4z6hO39nxhPO5S2t/3
dLT66mciUWbmT5utvrzH7+xs/MufiQx3SZj734BWeQD49f4DmhUpMw==
由于熵的缺乏真的很可怕,显然我在做一些非常错误的事情来得到这个结果,我在浪费带宽,或[=33]的序列=]'s 是有效信息。但这太可怕了。
发生的事情是,对于数据的第一部分,deflate 格式的压缩能力已经饱和。 deflate 中可能的最大压缩比是 1032:1.
在压缩后的数据中,前1,083,300字节全部为零。几乎所有这些都压缩到大约 1,000 个零字节。
一切顺利。你没有做错任何事。别担心。
我正在使用 Deflater
通过网络发送一些消息。
我正在使用:
public static byte[] compress(byte[] data,int level) throws IOException {
Instant starts = Instant.now();
Deflater deflater = new Deflater();
deflater.setInput(data);
deflater.setLevel(/*level*/9);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
deflater.finish();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer); // returns the generated code... index
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
System.out.println("Original: " + data.length / 1024 + " Kb");
System.out.println("Compressed: " + output.length / 1024 + " Kb");
Instant ends = Instant.now();
System.out.println("deflating time:"+Duration.between(starts, ends));
return output;
}
压缩通用字节数组,
和:
msg.setDataString(Base64.getEncoder().encodeToString(data));
将对象嵌入 xml。
有一段时间一切正常,但今天使用我看到的调试器:
msg.dataString
(这是一组几乎没有图像的 rgb 像素(3 字节)数组)
eNrt3e1i4jYQBdCR3/+hu+XTYM1I2IEku+fUbbdRDARMf1wmVxEAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAPCXan/++v/Iltvp6J8Z9bkAAAAAAPArtc1x/ur1z8/r58S8ncTmuP+7v9qKc8+n
3+/n+Rj9GAAAAAAA8HHt/odNrn2KuzP16jVKn7FsjqGojqiOMFcPAAAAAEClSJHLBPo2Ut7J
2x/n1F8N05fXg/bJYzaN7y4O0ngAAAAAAP5hRWBczoJHlU2fvrhEHa1fAvBuJr5c/2vZHO36
z+y4rG5PvX5D/Dk697rc8/beY4rxbHy5CgAAAADAL9dmJth7A9tVWn5thMmmxidmzR9D8Ids
e1mu4fombV9Oi2edzPx84mV5m7cvSyy9KP56W3H6hshn44vPEGJutYzizc0DAAAAAPxUtx1L
k0LycoR9ULBet7QsVRq/TETxS8c1he9rvZh98rjcczZ0P/qIYNSOE3nwPlEcf3stpfEAAAAA
AG9Vpq/5hPUgUp/ctXRYGdPL29uqH2Y7pN725u39JpnJvH0pEvWHfpt95fD1M51/7BFa4wEA
AAAAvkwZuLb8u+JWLJ7kvFHlvIcS9Wq+fXnoY++MkdfJ9mBGvU7Us2L4ien3slU+b6QfpPGD
Lvxhj/5l91phPAAAAADARZWSpuXfqyH1Tt5elsKst/kcjbfvG+ZOB8GrHU8Hafh6R9TOfPty
oFKm7n4vem4Gsfr4CYvRavSn4K9V+tHs0QoAAAAAsJpDz4fYW/ZdrdrXNKodPGdi3qm8vc7i
u6u3vVDTspm8UKYVM+qrID9to0kaY5aJufrBBPvM5rGd1WhHmnvyYvhrGU111QEAAAAA/Ep5
pF6F7ecZ9SRvP082L3FKbV/sfDmumuiu91Ldn7fXvTKrr2Z5fLa6erj9+faHR/98twd+FyCO
Z+0Tv7/Q2Q1X5g4AAAAA/FK3ALSTtterLS0UmdjUNPauRvtuWXJdz54XI+pFkU0b5+1TI+y9
Bpx3mauhyfqCRqevLkFNNQAAAADAh9X55GaLy1Vj+72Mu3ezt9VtGN/enebuy2vjQM5f5+1x
y8NbMTy/7Jv2Hybk1cz+O36T4N0vcH4VRUQrd+mt9/AFAAAAAP51bbDaBhO/STnMbSl6Q+rx
1PKxLf5IK7qPZ+b1yTun32OJj6TIO5rj39iu83lRPpkxlbe3yLbaXUXv2xqj+3p3D9+2Ljx6
upzv7wYAAAAA4O9Qz+UWm5a20Q0OTu6Gm9ns+331SF1MnbdHfnzX5PXuTwGO9Oesb7nzVKzW
OgH0tzxTh+73C57F+ncf0tViw4LLWykpVwIAAAAAfprdLRltbrV7j6th387JRej9mLe/GoxH
Of883/Bd32/1kF7dPfTrYuYqM09z4OTcde/K6DOR7rlvCeSjrBSKGP24R5/ifb83UV110Qbv
hZC9AwAAAMBnHemdbhPHy3fcZufb66R3V+59sGV9ZoZ5x/0eiZ9jVEh+K1rp5N7lapWox/A1
SpL1Ud4e4zb85O4vgXuyl+7h3Lvt+wwoDr2C1d4E6+e5/+6TxgMAAADAV6szt5m8/dXbnL/j
PL1Mht8nAvmsi6bNnbivFWRiSnl3IczOo/7Z00rxzVOfp/H1ZyLFK5h+1nL5O6n+v9z+5fpJ
utKTkvVRu/v8tbFjdbKnqLsUxdVzf0KzR5W9u5scHgAAAAB+jDaceG/5qa/Nz28z2Zmkd8dU
9nSzR5WK93PgfEp5kORHGdXGTKI+0fvT2yD0+aXqvopteJlk10b+mG4z6hO39nxhPO5S2t/3
dLT66mciUWbmT5utvrzH7+xs/MufiQx3SZj734BWeQD49f4DmhUpMw==
由于熵的缺乏真的很可怕,显然我在做一些非常错误的事情来得到这个结果,我在浪费带宽,或[=33]的序列=]'s 是有效信息。但这太可怕了。
发生的事情是,对于数据的第一部分,deflate 格式的压缩能力已经饱和。 deflate 中可能的最大压缩比是 1032:1.
在压缩后的数据中,前1,083,300字节全部为零。几乎所有这些都压缩到大约 1,000 个零字节。
一切顺利。你没有做错任何事。别担心。