来自 weather.gov 的当前观察提要被禁止 (403)

Question

Weather.gov 当前观察提要突然开始对来自 HTTPClient 的所有请求失败，同样我观察到互联网上的许多网站使用 AJAX 调用 weather.gov 也失败了。

所有调用 weather.gov 当前观察提要的结果，例如http://w1.weather.gov/xml/current_obs/TAPA.xml、return 一个 403。表示 URL 在浏览器中正确解析。

Answer 1

联系 weather.gov 得到了非常快的回复，即：

Applications accessing resources on weather.gov now need to provide a User-Agent header in any HTTP request. Requests without a user agent are automatically blocked. We have implemented this usage policy due to a small number of clients utilizing resources far in excess of what most would consider reasonable.

We recommend providing a user agent string in the following format:

ApplicationName/vX.Y (http://your.app.url/; contact.email@example.com)

This will both uniquely identify your application and allow us to contact you and work with you if we observe abnormal application behavior that may result in a block.

Please feel free to email us back if you continue to have issues after verifying that your application is sending the proper headers.

Thanks for using weather.gov.

=======

这是一段 C# 代码。关键是您需要创建请求对象，然后在调用之前向其附加自定义用户代理字符串。

...
var request = new HttpRequestMessage(HttpMethod.Post, httpClient.BaseAddress.AbsoluteUri);
request.Headers.Add("User-Agent", "MyApplication/v1.0 (http://foo.bar.baz; foo@bar.baz)");
var httpResponse = httpClient.SendAsync(request).Result;
...

希望这对大家有所帮助。干杯

Answer 2

这很好，但是当您使用 XMLDocument 并调用 Load() 时，您不能设置 "User-Agent" 像这样（这曾经有效）：

        XmlDocument doc = new XmlDocument();
        string stationName = @"http://w1.weather.gov/xml/current_obs/PASC.xml";
        doc.Load(stationName);  // exception 403 occurs here now

现在您需要执行 GET，然后将 User-Agent 设置为您的公司或电子邮件，然后使用 XmlDocument，例如：

       XmlDocument doc = new XmlDocument();

       string stationName = @"http://w1.weather.gov/xml/current_obs/PASC.xml";

        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(stationName);

        request.Method = "GET";
        request.ContentType = "application/x-www-form-urlencoded";
        request.UserAgent = "MyApplication/v1.0 (http://foo.bar.baz; foo@bar.baz)";
        request.Accept = "Accept=text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";

        HttpWebResponse response = (HttpWebResponse)request.GetResponse();
        Stream resStream = response.GetResponseStream();

        doc.Load(resStream);
        try
        {

            XmlNodeList list = doc.GetElementsByTagName("temp_f");
            if (list.Count > 0)
            {
                float fTemperature = float.Parse(list.Item(0).InnerText);

            }
        }

Answer 3

我遇到了同样的问题，但使用的是 PowerShell。这是使用 HttpWebRequest 设置用户代理的方法。

$weather_url = "http://w1.weather.gov/xml/current_obs/KLNK.xml"
$request = [System.Net.HttpWebRequest]::Create($weather_url)
$request.UserAgent = " {Enter your agent} "
$response = $request.GetResponse()

$doc = New-Object System.Xml.XmlDocument    
$doc.Load($response.GetResponseStream())

$temp_f = [int] $doc.current_observation.temp_f

Answer 4

如果你从 Étude 12-1 from Études for Elixir 中找到这个，这就是最终对我有用的东西

  s_url = "http://w1.weather.gov/xml/current_obs/" <> weather_station <> ".xml"
 :httpc.request(:get, { to_char_list(s_url),
                        [ {'User-Agent','ErlangEtudes/v1.0 (http://example.org;me@example.org)' } ] 
                      },[],[] )

Answer 5

这个答案对我有帮助，但实际上没有用。我仍然收到错误响应。

我在这一行中将 Post 更改为获取：

var request = new HttpRequestMessage(HttpMethod.Post...

收件人：

var request = new HttpRequestMessage(HttpMethod.Get...

然后，它运行良好。

来自 weather.gov 的当前观察提要被禁止 (403)

Current Observation feed from weather.gov forbidden (403)

xml

rss

weather

noaa