Ruby Net::HTTP 400 错误请求

Question

我正在尝试使用 http://www.nasdaqomxnordic.com/shares/listed-companies/nordic-large-cap 来抓取它的数据。这是我使用 Net::HTTP:

发送获取请求的代码

require 'net/http'
require 'uri'

def get_stocks()
    uri = URI.parse('http://www.nasdaqomxnordic.com/aktier/listed-companies/stockholm')
    response = Net::HTTP.get_response(uri)
    puts response
 end

get_stocks()

我测试过的其他网站运行良好，响应为 200:OK，但 http://www.nasdaqomxnordic.com/shares/listed-companies/nordic-large-cap returns 为 #<Net::HTTPBadRequest:0x00007ffe8f84ec30>，我不明白为什么。

更多上下文 response.body returns:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="TEXT/HTML; CHARSET=utf-8"/>
<title>400 Bad Request</title></head>
<body>
    <H2>400 Bad Request</H2>
    <p>The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.</p>
    <p>This page can't be displayed.<br/>The incident ID is: 10039581164792379.</p>
    <p>If you would like assistance, please contact the Support for additional information.<br></p>
</body>
</html>

我该怎么做才能获得 200:OK？

Answer 1

我认为您需要设置请求的 User-Agent 属性。以下代码有效。

require 'net/http'
require 'uri'

def get_stocks()
  uri = URI.parse("http://www.nasdaqomxnordic.com/shares/listed-companies/nordic-large-cap")
  http = Net::HTTP.new(uri.host, uri.port)
  request = Net::HTTP::Get.new(uri.request_uri)
  user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.86 Safari/537.36'
  request.initialize_http_header({"User-Agent" => user_agent})

  response = http.request(request)
  puts response.inspect 
 end

get_stocks() # #<Net::HTTPOK 200 OK readbody=true>

您可以使用 response.body

获取响应正文

Ruby Net::HTTP 400 错误请求

Ruby Net::HTTP 400 Bad Request

ruby

httpresponse

net-http