Spring WebClient 损坏二进制数据

Question

我正在将包含二进制数据的文件从服务 A 发送到服务 B。当文件数量相对较少时（比如 5 个）一切正常。但是，当我尝试发送更多文件（比如说几百个）时，有时会失败。我试图检查这个二进制数据发生了什么，看起来 WebClient 以某种方式破坏了它（奇怪的填充出现在末尾）。

我创建了一个最小的可重现示例来说明这个问题。

服务 B 中的端点（使用二进制文件）：

@RestController
class FilesController {

    @PostMapping(value = "/files")
    Mono<List<String>> uploadFiles(@RequestBody Flux<Part> parts) {
        return parts
                .filter(FilePart.class::isInstance)
                .map(FilePart.class::cast)
                .flatMap(part -> DataBufferUtils.join(part.content())
                        .map(buffer -> {
                            byte[] data = new byte[buffer.readableByteCount()];
                            buffer.read(data);
                            DataBufferUtils.release(buffer);
                            return Base64.getEncoder().encodeToString(data);
                        })
                )
                .collectList();
    }
}

说明服务 A 如何发送数据的测试：

public class BinaryUploadTest {

    private final CopyOnWriteArrayList<String> sentBytes = new CopyOnWriteArrayList<>();

    @BeforeEach
    void before() {
        sentBytes.clear();
    }

    /**
     * this test passes all the time
     */
    @Test
    void shouldUpload5Files() {
        // given
        MultiValueMap<String, HttpEntity<?>> body = buildResources(5);

        // when
        List<String> receivedBytes = sendPostRequest(body);

        // then
        assertEquals(sentBytes, receivedBytes);
    }

    /**
     * this test fails most of the time
     */
    @Test
    void shouldUpload1000Files() {
        // given
        MultiValueMap<String, HttpEntity<?>> body = buildResources(1000);

        // when
        List<String> receivedBytes = sendPostRequest(body);

        // then
        assertEquals(sentBytes, receivedBytes);
    }

    private List<String> sendPostRequest(MultiValueMap<String, HttpEntity<?>> body) {
        return WebClient.builder().build().post()
                .uri("http://localhost:8080/files")
                .contentType(MediaType.MULTIPART_FORM_DATA)
                .body(BodyInserters.fromMultipartData(body))
                .retrieve()
                .bodyToMono(new ParameterizedTypeReference<List<String>>() {
                })
                .block();
    }

    private MultiValueMap<String, HttpEntity<?>> buildResources(int numberOfResources) {
        MultipartBodyBuilder builder = new MultipartBodyBuilder();
        for (int i = 0; i < numberOfResources; i++) {
            builder.part("item-" + i, buildResource(i));
        }
        return builder.build();
    }

    private ByteArrayResource buildResource(int index) {
        byte[] bytes = randomBytes();
        sentBytes.add(Base64.getEncoder().encodeToString(bytes)); // keeps track of what has been sent
        return new ByteArrayResource(bytes) {
            @Override
            public String getFilename() {
                return "filename-" + index;
            }
        };
    }

    private byte[] randomBytes() {
        byte[] bytes = new byte[ThreadLocalRandom.current().nextInt(16, 32)];
        ThreadLocalRandom.current().nextBytes(bytes);
        return bytes;
    }
}

数据损坏的原因可能是什么？

Answer 1

原来是 Spring 框架中的一个错误（更准确地说是 MultipartParser class 中）。我创建了一个 GitHub issue which will be fixed in the next version (5.3.16). The bug is fixed by this commit.

Spring WebClient 损坏二进制数据

Spring WebClient corrupts binary data

java

spring

project-reactor

reactor-netty

spring-webclient