如何将使用 Rusoto 从 S3 下载的文件保存到我的硬盘？

Question

我正在尝试使用 Rusoto 从存储桶下载文件，我正在获取文件内容：

fn get_object(client: &TestClient, bucket: &str, filename: &str) {
    let get_req = GetObjectRequest {
        bucket: bucket.to_owned(),
        key: filename.to_owned(),
        ..Default::default()
    };

    let result = client.get_object(&get_req).sync().expect("Couldn't GET object");


    let stream = result.body.unwrap();
    let body = stream.concat2().wait().unwrap();

    assert!(body.len() > 0);
}

如何将这个 GetObjectOutput(result) 对象保存到文件中？

Answer 1

你快到了。您的代码会将对象放入 body，即 Vec<u8>.

要将 body 的内容写入文件：

use std::io::Write;
use std::fs::File;

let mut file = File::create("/path/to/my-object").expect("create failed");
file.write_all(&body).expect("failed to write body");

Answer 2

Rusoto 现在使用标准库期货，不再提供 sync 方法，因此之前的答案不再有效。

读入记忆

use futures::stream::TryStreamExt;
use rusoto_core::Region;
use rusoto_s3::{GetObjectRequest, S3Client, S3};

type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;

const BUCKET_NAME: &str = "my very own bucket name";

#[tokio::main]
async fn main() -> Result<()> {
    let client = S3Client::new(Region::UsEast2);

    let mut object = client
        .get_object(GetObjectRequest {
            bucket: BUCKET_NAME.into(),
            ..Default::default()
        })
        .await?;

    let body = object.body.take().expect("The object has no body");

    let body = body.map_ok(|b| b.to_vec()).try_concat().await?;
    println!("body length: {}", body.len());

    Ok(())
}

需要指定

AWS_ACCESS_KEY_ID 和 AWS_SECRET_ACCESS_KEY。我选择在代码之外设置环境变量。

流式传输到文件

use rusoto_core::Region;
use rusoto_s3::{GetObjectRequest, S3Client, S3};
use tokio::{fs::File, io};

type Error = Box<dyn std::error::Error>;
type Result<T, E = Error> = std::result::Result<T, E>;

const BUCKET_NAME: &str = "my very own bucket name";

#[tokio::main]
async fn main() -> Result<()> {
    let client = S3Client::new(Region::UsEast2);

    let mut object = client
        .get_object(GetObjectRequest {
            bucket: BUCKET_NAME.into(),
            ..Default::default()
        })
        .await?;

    let body = object.body.take().expect("The object has no body");

    let mut body = body.into_async_read();
    let mut file = File::create("/tmp/a-place-to-write").await?;
    io::copy(&mut body, &mut file).await?;

    Ok(())
}

虽然 ByteStream 有一个诱人的 into_blocking_read method, I do not recommend using it. If you attempt to use it inside of an async context, you get a panic because it starts a nested Tokio executor. If you use it outside of an async context, it will truncate the data 除非你非常小心地使用异步运行时而不是在中。

另请参阅：

依赖版本

[dependencies]
rusoto_s3 = "0.43.0"
rusoto_core = "0.43.0"
tokio = { version = "0.2.21", features = ["macros"] }
futures = "0.3.5"

如何将使用 Rusoto 从 S3 下载的文件保存到我的硬盘？

How to save a file downloaded from S3 with Rusoto to my hard drive?

file

amazon-s3

file-transfer

rust

rusoto

读入记忆

流式传输到文件

依赖版本