使用 Amazon S3 使用 Nginx 限制带宽

Question

我在 Amazon S3 上托管了很大的下载文件（有些超过 5 GB）。我的主服务器是 Nginx。 Amazon S3 没有 public 访问权限。文件带有签名的 URLs。

使用 Amazon S3 时是否有限制带宽的方法？我知道在 Amazon S3 上没有选项，但我们可以使用 Nginx 作为代理并从那里创建它吗？

我正在尝试使用 link 中的示例：

https://coderwall.com/p/rlguog/nginx-as-proxy-for-amazon-s3-public-private-files

此代码块：

location ~* ^/proxy_private_file/(.*) {
  set $s3_bucket        'your_bucket.s3.amazonaws.com';
  set $aws_access_key   'AWSAccessKeyId=YOUR_ONLY_ACCESS_KEY';
  set $url_expires      'Expires=$arg_e';
  set $url_signature    'Signature=$arg_st';
  set $url_full         '$aws_access_key&$url_expires&$url_signature';

  proxy_http_version     1.1;
  proxy_set_header       Host $s3_bucket;
  proxy_set_header       Authorization '';
  proxy_hide_header      x-amz-id-2;
  proxy_hide_header      x-amz-request-id;
  proxy_hide_header      Set-Cookie;
  proxy_ignore_headers   "Set-Cookie";
  proxy_buffering        off;
  proxy_intercept_errors on;

  resolver               172.16.0.23 valid=300s;
  resolver_timeout       10s;

  proxy_pass             http://$s3_bucket$url_full;

}

我不明白的是如何将创建的签名 URL 从 PHP 传递到 Nginx 配置？所以我可以告诉 Nginx 去那个签名的 URL 作为代理。

Answer 1

我找到了解决方案。这是：

首先在 nginx 配置中打开 http 块。我们将创建限制每个 IP 连接所需的区域。

limit_conn_zone $binary_remote_addr zone=addr:10m;

现在在/etc/nginx/conf.d/sitename.conf 或您定义它的任何地方打开您的服务器块。创建一个内部位置。我们会将 PHP 请求重定向到此处：

location ~* ^/internal_redirect/(.*?)/(.*) {
# Do not allow people to mess with this location directly
# Only internal redirects are allowed
internal;

# Location-specific logging, so we can clearly see which requests
# passing through proxy and what is happening there
access_log /var/log/nginx/internal_redirect.access.log main;
error_log /var/log/nginx/internal_redirect.error.log warn;

# Extract download url from the request
set $download_uri ;
set $download_host ;

# Extract the arguments from request.
# That is the Signed URL part that you require to get the file from S3 servers
if ($download_uri ~* "([^/]*$)" ) {
    set  $filename  ;
}

# Compose download url
set $download_url $download_host/$download_uri?$args;

# Set download request headers
proxy_http_version      1.1;
proxy_set_header        Connection "";
proxy_hide_header       x-amz-id-2;
proxy_hide_header       x-amz-request-id;
proxy_hide_header       Set-Cookie;
proxy_ignore_headers    "Set-Cookie";

# Activate the proxy buffering, without it limiting bandwidth speed in proxy will not work!
proxy_buffering on;

# Buffer 512 KB data
proxy_buffers 32 16k;

proxy_intercept_errors  on;
resolver        8.8.8.8 valid=300s;
resolver_timeout        10s;

# The next two lines could be used if your storage
# backend does not support Content-Disposition
# headers used to specify file name browsers use
# when save content to the disk
proxy_hide_header Content-Disposition;
add_header Content-Disposition 'attachment; filename="$filename"';

# Do not touch local disks when proxying
# content to clients
proxy_max_temp_file_size 0;

# Limit the connection to one per IP address
limit_conn addr 1;

# Limit the bandwidth to 300 kilobytes
proxy_limit_rate 300k;

# Set logging level to info so we can see everything.
# All levels you can set: info | notice | warn | error
limit_conn_log_level info;   

# Finally download the file and send it to client
# Beware that you can shouldn't include "htttp://" or "https://"
# in proxy. Doing that will cause an "invalid port in upstream" error.
proxy_pass $download_url;
}

在PHP中画龙点睛并将您签名的URL发送到Nginx：

header( 'X-Accel-Redirect: ' . '/internal_redirect/' . $YOUR_SIGNED_URL );

使用 Amazon S3 使用 Nginx 限制带宽

Restricting bandwidth with Nginx using Amazon S3

nginx

amazon-s3

bandwidth