Spring 批处理/Azure 存储帐户 blob 资源 [container"foo", blob='bar'] 无法解析为绝对文件路径

Question

我正在尝试使用 Spring 写入 Azure 存储。

我正在 Bean 中配置资源，而不是从 Class 自动装配它。

BlobServiceClient client = new BlobServiceClientBuilder().connectionString(connectionString).endpoint(endpoint).buildClient();

这是 Azure 存储容器中的 blob 名称：

String resourceString = "azure-blob://foo/bar.csv"

设置 Azure 存储模式解析器：

AzureStorageResourcePatternResolver storageResourcePatternResolver = new AzureStorageResourcePatternResolver(client);

然后获取资源：

Resource resource = storageResourcePatternResolver.getResource(resourceString);

这是我遇到的错误：

java.lang.UnsupportedOperationException: Azure storage account blob resource [container='foo', blob='bar.csv'] cannot be resolved to absolute file path
at com.azure.spring.autoconfigure.storage.resource.BlobStorageResource.getFile(BlobStorageResource.java:83) ~[azure-spring-boot-3.4.0.jar:na]
at org.springframework.batch.item.support.AbstractFileItemWriter.getOutputState(AbstractFileItemWriter.java:367) ~[spring-batch-infrastructure-4.3.2.jar:4.3.2]
at org.springframework.batch.item.support.AbstractFileItemWriter.open(AbstractFileItemWriter.java:308) ~[spring-batch-infrastructure-4.3.2.jar:4.3.2]
at org.springframework.batch.item.support.AbstractFileItemWriter$$FastClassBySpringCGLIB$$f2d35c3.invoke(<generated>) ~[spring-batch-infrastructure-4.3.2.jar:4.3.2]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692) ~[spring-aop-5.3.5.jar:5.3.5]
at org.springframework.batch.item.file.FlatFileItemWriter$$EnhancerBySpringCGLIB$ce6a900.open(<generated>) ~[spring-batch-infrastructure-4.3.2.jar:4.3.2]
at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:104) ~[spring-batch-infrastructure-4.3.2.jar:4.3.2]
at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:311) ~[spring-batch-core-4.3.2.jar:4.3.2]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:205) ~[spring-batch-core-4.3.2.jar:4.3.2]
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler.call(TaskExecutorPartitionHandler.java:138) [spring-batch-core-4.3.2.jar:4.3.2]
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler.call(TaskExecutorPartitionHandler.java:135) [spring-batch-core-4.3.2.jar:4.3.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_221]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_221]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_221]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_221]2021-05-25 09:24:36.168  INFO 4916 --- [ taskExecutor-3] o.s.batch.core.step.AbstractStep         : Step: [slaveStep:partition155] executed in 347ms2021-05-25 09:24:36.240 ERROR 4916 --- [ taskExecutor-1] o.s.batch.core.step.AbstractStep         : Encountered an error executing step slaveStep in job job1

非常感谢任何帮助。

Answer 1

searchLocation 应以 azure-blob:// 或 azure-file:// 开头。您评论中的“blob”不正确。

azure-blob://foo/bar.csv 表示 "bar.csv" blob 在 "foo" 容器中。请检查您的存储，确保 blob 存在。

例如，我的blob URL是https://pamelastorage123.blob.core.windows.net/pamelac/test.txt，所以azure-blob://pamelac/test.txt是对的。

StorageExampleApplication.java:

@SpringBootApplication
public class StorageExampleApplication {

    public static void main(String[] args) {
        SpringApplication.run(StorageExampleApplication.class, args);
        testRescource();
    }

    public static void testRescource(){
        String searchLocation = "azure-blob://<container-name>/<blob-name>";
        String connectionString = "<Connection string>";
        String endpoint = "https://<account-name>.blob.core.windows.net";

        BlobServiceClient client = new BlobServiceClientBuilder().connectionString(connectionString).endpoint(endpoint).buildClient();
        AzureStorageResourcePatternResolver storageResourcePatternResolver = new AzureStorageResourcePatternResolver(client);

        Resource resource = storageResourcePatternResolver.getResource(searchLocation);
        System.out.println(resource.getFilename());
    }

}

application.properties:

azure.storage.account-name=[storage-account-name]
azure.storage.account-key=[storage-account-access-key]
azure.storage.blob-endpoint=[storage-blob-endpoint-URL]

结果：

Answer 2

FlatFileItemWriter 旨在处理代表可写文件的任何资源。以下是 Javadoc 的摘录：

The location of the output file is defined by a Resource and must represent
a writable file.

本编写器的初始化过程通过调用Resource#getFile检查传递的资源是否为可写文件。在您的情况下，您正在使用的 Resource 实现，即 BlobStorageResource 为此操作抛出 java.lang.UnsupportedOperationException，因此您的错误。您需要提供一个 Resource 实现来代表 FlatFileItemWriter.

预期的文件

Spring 批处理/Azure 存储帐户 blob 资源 [container"foo", blob='bar'] 无法解析为绝对文件路径

Spring Batch / Azure Storage account blob resource [container"foo", blob='bar'] cannot be resolved to absolute file path

spring

azure

azure-storage

spring-batch

azure-blob-storage