如何在 Ruby 中通过 HTTP 下载延迟文件？

Question

我使用以下 Ruby 函数通过 HTTP 下载各种文件：

def http_download(uri, filename)
  bytes_total = nil
  begin
    uri.open(
      read_timeout: 500,
      content_length_proc: lambda { |content_length|
        bytes_total = content_length
      },
      progress_proc: lambda { |bytes_transferred|
        if bytes_total
          print("\r#{bytes_transferred} of #{bytes_total} bytes")
        else
          print("\r#{bytes_transferred} bytes (total size unknown)")
        end
      }
    ) do |file|
      open filename, 'w' do |io|
        file.each_line do |line|
          io.write line
        end
      end
    end
  rescue => e
    puts e
  end
end

我也想下载文件(csv, kml, zip, geojson) from this website。但是，设置了某种延迟。当我点击下载link浏览器需要一点时间才会出现下载 window。我想文件需要在服务器上进行处理才能提供。

如何修改我的脚本以将延迟考虑在内？

我是运行Ruby2.2.2.

Answer 1

这里是根据post和注释修改：

require 'open-uri'

def http_download(uri, filename)
  bytes_total = nil
  index = 1
  begin
    open(
      uri,
      read_timeout: 500,
      content_length_proc: lambda { |content_length|
        bytes_total = content_length
      },
      progress_proc: lambda { |bytes_transferred|
        if bytes_total
          print("\r#{bytes_transferred} of #{bytes_total} bytes")
        else
          print("\r#{bytes_transferred} bytes (total size unknown)")
        end
      }
    ) do |io|
      # if "application/json" == io.content_type
      if io.is_a? StringIO
        raise " --> Failed, server is processing. Retry the request ##{index}"
      else # Tempfile
        puts "\n--> Succeed, writing to #{filename}"
        File.open(filename, 'w'){|wf| wf.write io.read}
      end
    end
  rescue => e
    puts e
    return if e.is_a? OpenURI::HTTPError # Processing error

    index += 1
    return if index > 10

    sleep index and retry
  end
end

如何在 Ruby 中通过 HTTP 下载延迟文件？

How to download a delayed file via HTTP in Ruby?

ruby

download

delay

open-uri