Java 使用 Spring RestTemplate 上传 S3

Question

我想使用 SpringBoot RestTemplate 进行此调用以将文件上传到 S3 存储桶：https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

PUT /my-image.jpg HTTP/1.1
Host: myBucket.s3.<Region>.amazonaws.com
Date: Wed, 12 Oct 2009 17:50:00 GMT
Authorization: authorization string
Content-Type: text/plain
Content-Length: 11434
x-amz-meta-author: Janet
Expect: 100-continue
[11434 bytes of object data]

和

    @Bean
    public RestTemplate restTemplate(RestTemplateBuilder builder) {
        return builder.rootUri("")
                .additionalInterceptors((request, body, execution) -> {
                    request.getHeaders().add("Authorization",
                            "Bearer a0d78d7922f333ee22d75bea53d01hhkjk83f5ac03f11ccd87787");
                    return execution.execute(request, body);
                }).build();
    }

我试过了

Resource resource = new ClassPathResource("logback.xml");
    HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.TEXT_PLAIN);

    HttpEntity<byte[]> requestEntity
            = new HttpEntity<>(StreamUtils.copyToByteArray(resource.getInputStream()), headers);

    Map<String, Object> parameters = new HashMap<>(4);
    parameters.put("cors_enabled", true);
    parameters.put("acl", "private");
    parameters.put("key", "my-key");
    parameters.put("Bucket", "parameters.put("Bucket", "https://cloud.linode.com/object-storage/buckets/eu-central-1/my-bucket-2020");");

    restTemplate.put("https://api.linode.com/v4/object-storage/buckets", requestEntity, parameters);

但我得到了

org.springframework.web.client.HttpClientErrorException$MethodNotAllowed: 405 METHOD NOT ALLOWED: [{"errors": [{"reason": "Method Not Allowed"}]}]

获取时也有问题：

MultiValueMap<String, Object> body
            = new LinkedMultiValueMap<>();

    UriComponentsBuilder builder =
            UriComponentsBuilder.fromHttpUrl("https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url");
    builder.queryParam("method", "GET");
    builder.queryParam("name", "43f959d9-a11a-4f2cec88fd7e.JPG");

    body.add("method", "GET");
    body.add("name", "43f959d9-a11a-4f2cec88fd7e.JPG");

    HttpHeaders headers = new HttpHeaders();

    HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body, headers);

    restTemplate.postForEntity(builder.build().encode().toUri(),
            requestEntity, LinodeResponse.class);

和响应：

org.springframework.web.client.HttpClientErrorException$BadRequest: 400 BAD REQUEST: [{"errors": [{"reason": "name is required", "field": "name"}, {"reason": "method is required", "field": "method"}]}]

ans 使用 AWS-SDK 访问时出现此错误：

com.amazonaws.services.s3.model.AmazonS3Exception: The AWS Access Key Id you provided does not exist in our records.

Answer 1

Spring 相当于您提供的 cURL 命令可以是：

RestTemplate restTemplate  = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
String token = "";
headers.set(HttpHeaders.AUTHORIZATION, token);

JSONObject data = new JSONObject();
data.put("cors_enabled", true);
data.put("acl", "private");

HttpEntity<String> requestEntity = new HttpEntity<String>(data.toString(), headers);

String url = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/bonansa15122020/access";
HttpEntity<String> responseEntity = restTemplate.exchange(url, HttpMethod.PUT, requestEntity, String.class);

在您的第一个示例中，您没有提供授权 header，因此您会收到 401 响应。您没有使用您在此处使用 RestTemplateBuilder 创建的 RestTemplate。

在你的第二个例子中，请求 body 似乎不是 JSON（你正在读取 logback 文件，所以里面不太可能有 JSON） .看来 Linode API 需要一个 JSON body.

Answer 2

更新:

我相信您可以使用 PUT 请求作为 POST 对端点 https://api.linode.com/v4/object-storage/buckets/{clusterId}/{bucket}/object-url

请求的一部分

此处有更多详细信息 - https://developers-linode.netlify.app/api/v4/object-storage-buckets-cluster-id-bucket-object-url#post

我将无法测试，因为我没有 linode 帐户。

我认为另一个可行的解决方案是使用适用于 s3 的 aws sdk 将文件上传到 linode 端点。

这是一个简单的例子 - https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/java/example_code/s3/src/main/java/aws/example/s3/PutObject.java

原文：

根据 linode api 文档，方法应该是 post.

https://www.linode.com/docs/api/object-storage/#object-storage-bucket-create

restTemplate.post("https://api.linode.com/v4/object-storage/buckets", requestEntity, parameters)

另请检查请求正文以符合文档。

Answer 3

Linode 似乎提供了一个 API to generate presigned urls 用于与 S3 中的对象交互。

要使用 API，首先，您可以创建两个 POJO，代表我们将从 API 发送和接收的请求和响应，这样我们就可以用来序列化和反序列化 JSON 信息.

对于请求对象：

public class LinodeGeneratePresignedUrlRequest {
  private String method;

  private String name;

  @JsonProperty("content_type")
  private String contentType;

  @JsonProperty("expires_in")
  private int expiresIn;

  // Getters and setters
}

对于响应：

pubic class LinodeGeneratePresignedUrlResponse {

  private String url;

  // Getters and setters
}

这些对象与 endpoint 所需的信息相匹配。

如果您想使用 Linode API 在您的存储桶中创建一个对象，您首先需要请求一个预签名 URL。获得后，您将使用此 URL 对存储桶对象执行实际操作。该操作由传递给 API 的 method 参数定义。考虑以下示例：


// Obtain a reference to the RestTemplate instance.
// It should support the interchange of JSON information
RestTemplate restTemplate  = new RestTemplate();

HttpHeaders headers = new HttpHeaders();

// Set content type to the one required by the Linode API application/json
headers.setContentType(MediaType.APPLICATION_JSON);

// Set the appropriate credentials for the Linode API
String token = "your token";
headers.set(HttpHeaders.AUTHORIZATION, "Bearer" + token);

// Create the presigned url request
LinodeGeneratePresignedUrlRequest linodeGeneratePresignedUrlRequest =
  new LinodeGeneratePresignedUrlRequest();
// Operation to perform when you interact with AWS later
// In this case, PUT because you need to create a new object
linodeGeneratePresignedUrlRequest.setMethod("PUT"); 
// The object name: can match or not the actual file you want to upload
linodeGeneratePresignedUrlRequest.setName("my-object-name.pdf"); 
// As you are performing an upload (PUT, POST), indicate the content type of 
// the information you are uploading to AWS. It should match the provided later
// when you interact with AWS. For instance, consider that you are uploading a PDF file
linodeGeneratePresignedUrlRequest.setContentType("application/pdf");
// Optionally, you can set the expiration time of the generated presigned url
// By default, an hour (3600 seconds)

// Perform the actual Linode API invocation
HttpEntity<LinodeGeneratePresignedUrlRequest> requestEntity = 
  new HttpEntity<LinodeGeneratePresignedUrlRequest>(linodeGeneratePresignedUrlRequest, headers);

// The Linode API URL for your cluster and bucket
String linodeApiUrl = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url";

HttpEntity<LinodeGeneratePresignedUrlResponse> responseEntity = restTemplate.exchange(linodeApiUrl, HttpMethod.POST, requestEntity, LinodeGeneratePresignedUrlResponse.class);

// Linde wil provide a response with a property named 'url' corresponding 
// to the presigned url that we can use to interact with AWS S3
LinodeGeneratePresignedUrlResponse linodeGeneratePresignedUrlResponse = responseEntity.getBody();

String signedUrl = linodeGeneratePresignedUrlResponse.getUrl();

// Now, send the actual file.
// I am following the example provided in the AWS documentation:
// https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObjectJavaSDK.html adapt for RestTemplate

HttpHeaders headersForS3 = new HttpHeaders();
// You should provide the same content type you indicated previously
headersForS3.set("Content-Type", "application/pdf");
Resource resource = new FileSystemResource("my-object-name.pdf"); 

HttpEntity<byte[]> requestEntityForS3 =
  new HttpEntity<>(
    StreamUtils.copyToByteArray(resource.getInputStream()), headersForS3);
// You should use the same HTTP verb as indicated in 
// the 'method' parameter before
restTemplate.exchange(signedUrl, HttpMethod.PUT, requestEntityForS3, Void.class);

检索创建的对象的过程非常相似：

RestTemplate restTemplate  = new RestTemplate();

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);

String token = "your token";
headers.set(HttpHeaders.AUTHORIZATION, "Bearer" + token);

LinodeGeneratePresignedUrlRequest linodeGeneratePresignedUrlRequest =
  new LinodeGeneratePresignedUrlRequest();
// Instead of PUT, indicate that you want to retrieve the object
linodeGeneratePresignedUrlRequest.setMethod("GET");
 // your object name
linodeGeneratePresignedUrlRequest.setName("my-object-name.pdf");

HttpEntity<LinodeGeneratePresignedUrlRequest> requestEntity = 
  new HttpEntity<LinodeGeneratePresignedUrlRequest>(linodeGeneratePresignedUrlRequest, headers);

String linodeApiUrl = "https://api.linode.com/v4/object-storage/buckets/eu-central-1/my-bucket-2020/object-url";

HttpEntity<LinodeGeneratePresignedUrlResponse> responseEntity = restTemplate.exchange(linodeApiUrl, HttpMethod.POST, requestEntity, LinodeGeneratePresignedUrlResponse.class);

LinodeGeneratePresignedUrlResponse linodeGeneratePresignedUrlResponse = responseEntity.getBody();

String signedUrl = linodeGeneratePresignedUrlResponse.getUrl();

// Read the object from your bucket
byte[] objectBytes = restTemplate.getForObject(signedUrl, byte[].class);
// And use the information as you need
Files.write(Paths.get("my-object-name.pdf"), objectBytes);

当然，如果 Linode 为您提供了适当的凭据，您也可以使用 AWS SDK 直接与 S3 进行交互。

Java 使用 Spring RestTemplate 上传 S3

Java S3 upload using Spring RestTemplate

amazon-s3

bucket

linode

resttemplate

spring-boot