使用 HttpWebRequest 和 HttpWebResponse 会抛出错误，但仅在某些网站上

Question

此函数检查 url 是否为有效地址，并与 http://www.google.com

一起使用

public static bool CheckValidURL(string url)
{
    try
    {
        //Creating the HttpWebRequest
        HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
        //Setting the Request method HEAD, you can also use GET too.
        request.Method = "GET";
        //Getting the Web Response.
        HttpWebResponse response = request.GetResponse() as HttpWebResponse;
        //Returns TRUE if the Status code == 200
        bool isValid;
        if (response.StatusCode == HttpStatusCode.OK)
        {
            isValid = true;
        }
        else
        {
            isValid = false;
        }
        response.Close();
        return isValid;
    }
    catch
    {
        //Any exception will returns false.
        return false;
    }
}

此代码在单击菜单项时调用，旨在更新卡片价格。它在 foreach 循环中，其中变量 c 属于自定义 class MagicCard.

string url = "";
string webCardName = c.name.ToLower().Replace(" ", "-").Replace(",", "").Replace("\'", "");
string webSetName = MagicCard.GetSetName(c).ToLower().Replace(" ", "-").Replace(",", "").Replace("\'", "");
url = string.Format("shop.tcgplayer.com/magic/{0}/{1}", webSetName, webCardName);
if (WebScraper.CheckValidURL(url) == false)
{
    MessageBox.Show("ERROR: URL = " + url);
    return;
}

最终 url 的一个示例是 this 虽然这是一个有效地址，但它被检测为无效，任何其他卡都会产生类似的 url不行。为什么会这样？

Answer 1

您的网站需要 User-Agent。我添加了这一行

request.UserAgent = "SO/1.0";

添加到您的代码中并且有效....

使用 HttpWebRequest 和 HttpWebResponse 会抛出错误，但仅在某些网站上

Using HttpWebRequest and HttpWebResponse throws an error, however only on certain websites

c#

httpwebrequest