将响应读入字节需要很长时间

Question

我正在尝试使用 reqwest 下载文件。响应状态为 200。当我尝试将响应读取为字节 response.bytes().await? 时，它会永远等待。

但是，当我尝试对相同的 URL 发出 curl 请求时，它通过了，我能够成功下载文件。

我不确定哪里出了问题或我应该如何从这里进行调试。欢迎任何建议。

use anyhow::Context;
use reqwest_middleware::ClientBuilder;
use reqwest_retry::{policies::ExponentialBackoff, RetryTransientMiddleware};
use reqwest_tracing::TracingMiddleware;
use std::fs;
use std::io::Cursor;
use std::os::unix::fs::PermissionsExt;
use std::path::PathBuf;
use std::{
    path::Path,
    process::{Command, Output},
};
        
async fn download() -> Result<(), anyhow::Error> {
            let panda_link = format!(
                "https://surya.jfrog.io/artifactory/binaries/panda/v2.34.0/{}/panda",
                ARCH_NAME
            );


            let retry_policy = ExponentialBackoff::builder().build_with_max_retries(3);
            let client = ClientBuilder::new(reqwest::Client::new())
                .with(TracingMiddleware)
                .with(RetryTransientMiddleware::new_with_policy(retry_policy))
                .build();
            println!("the client has been successfully built");

            let response = client
                .get(panda_link)
                .send()
                .await?;

            println!("got the response {}", response.status());
            response
                .error_for_status_ref()
                .context("Failed to download panda")?;
            println!("check if response was error is done");

            let response_bytes = response.bytes().await?;
            println!("reading response bytes");

            let mut content = Cursor::new(response_bytes);
            println!("reading bytes");

            let new_file_path = PathBuf::from("/");
            println!("this is the newfile_path {:?}", new_file_path);

            let mut file = std::fs::File::create(&new_file_path)
                .context(format!("Failed creating file {}", &new_file_path.display()))?;
            fs::set_permissions(&new_file_path, fs::Permissions::from_mode(0o750)).context(
                format!("Failed making {} executable", &new_file_path.display()),
            )?;
            println!("file created");
            std::io::copy(&mut content, &mut file)
                .context(format!("Failed saving file {}", &new_file_path.display()))?;

            Ok(())
}

Answer 1

我认为您遇到的问题是多种因素的结合：

个人互联网下载速度。
网站速度本身。（网站的流量负载，网站的位置，网站生成下载的速度）。
内存速度。
磁盘速度。

我用下面的代码做了一些测试。（它很实用，不会“永远等待”）

如果我将下载 link 指向 google 文档上的一个大文件（~100 Mb），它会在（给定或需要）一秒钟内下载，但如果我将它指向如果网站上的大文件 (~100 Mb) 速度不是很快，则需要几秒钟。

    // Download link
    let panda_link = format!(
        "https://surya.jfrog.io/artifactory/binaries/panda/v2.34.0/{}/panda",
        ARCH_NAME
    );

    // File Path
    let new_file_path = PathBuf::from("/");

    // Gets the response from website as bytes (Not sure if its stored in the memory. If it is stored in the memory than it would be impossible to download something that is larger than your total ram size.)
    let content = reqwest::get(panda_link).await?.bytes().await?;
    // Writes the bytes to the disk.
    fs::write(new_file_path, content)?;

将响应读入字节需要很长时间

Reading response into bytes takes forever

rust

reqwest