如何使用 webrequest 检查 link 是否损坏或关闭？

Question

我正在从新闻网站的 RSS 获取数据并显示在列表视图中。（例如标题、描述、发布日期、标签、图像和评论计数）但是当 link 损坏或关闭时我没有什么问题，我我正在接受错误。（mscorlib.dll 中发生了 'System.AggregateException' 类型的未处理异常）我想检查 link 是否已损坏或关闭 webrequest 跳过此地址并继续访问其他地址。

首先检查有关 webrequest 的主题，但我没有找到我想要的答案。只是这个主题给出了一个想法，但我没有集成到我的代码中。 (how to send web request to check the link whether the link is broken or not)

感谢您现在的帮助。

这是我的主class

public static Dictionary<string, HaberYildizi> HaberYildiziHaberList = new Dictionary<string, HaberYildizi>();
    public static bool degismisMi = false;

    public Dictionary<string, HaberYildizi> GetTagsHaberYildizi()
    {
        List<IBaseClass> yeniHaberList = new List<IBaseClass>();
        degismisMi = false;

        XmlDocument xdoc = new XmlDocument();
        xdoc.Load("http://www.haberyildizi.com/rss.php");
        XmlElement el = (XmlElement)xdoc.SelectSingleNode("/rss");

        if (el != null)
            el.ParentNode.RemoveChild(el);

        XmlNode Haberler = el.SelectSingleNode("channel");

        List<HaberYildizi> newHaberYildizi = new List<HaberYildizi>();
        string htmlStr = String.Empty;

        foreach (XmlNode haber in Haberler.SelectNodes("item"))
        {
            var link = haber.SelectSingleNode("link").InnerText;

            if (HaberYildiziHaberList.ContainsKey(link))
                continue;

            HaberYildizi haberYildizi = new HaberYildizi();
            haberYildizi.Title = WebUtility.HtmlDecode(haber.SelectSingleNode("title").InnerText);

            if (haber.SelectSingleNode("description").InnerText.Contains("</a>"))
            {
                var str1 = haber.SelectSingleNode("description").InnerText.IndexOf("</a>");
                var str2 = haber.SelectSingleNode("description").InnerText.Substring(str1 + 4);
                haberYildizi.Description = WebUtility.HtmlDecode(str2);
            }
            else
            {
                haberYildizi.Description = WebUtility.HtmlDecode(haber.SelectSingleNode("description").InnerText);
            }

            haberYildizi.Image = haber.SelectSingleNode("image").InnerText;

            haberYildizi.Link = link;
            var format = DateTime.Parse(haber.SelectSingleNode("pubDate").InnerText.Replace("Z", ""));
            haberYildizi.PubDate = format;

        //*************************************
            htmlStr = Utilities.GetResponseStr(haberYildizi.Link).Result; // mistake is here 
        //*************************************
            haberYildizi.Tags = GetTags(htmlStr);

            haberYildizi.Comment = GetCommentCount(htmlStr);

            if (HaberYildiziHaberList.ContainsKey(haber.SelectSingleNode("link").InnerText) == false)
            {
                degismisMi = true;
                HaberYildiziHaberList.Add(link, haberYildizi);
                newHaberYildizi.Add(haberYildizi);
            }
            yeniHaberList.Add(haberYildizi);
        }
        return HaberYildiziHaberList;
    }
    public int GetCommentCount(string htmlStr)
    {
        int count = 0;
        string commentKeyword = "label label-important\">";
        var comment = htmlStr.IndexOf(commentKeyword) + 23;
        if (comment != -1)
        {
            var comment2 = htmlStr.IndexOf("</span>", comment);

            var comment4 = htmlStr.Substring(comment, comment2 - comment).Trim();
            count = Convert.ToInt32(comment4);

        }
        return count;
    }
    public List<string> GetTags(string htmlStr)
    {            
        List<string> listele = new List<string>();
        string begenningKeyword = "<meta name=\"keywords\" content=\"";

        var tags = htmlStr.IndexOf(begenningKeyword);

        var final_response2 = htmlStr.Substring(tags + begenningKeyword.Length);
        var tagsBol = final_response2.IndexOf("\" />");

        var lastTags = final_response2.Substring(0, tagsBol);
        var tagsSonuc = lastTags.Split(',');
        foreach (var tag in tagsSonuc)
        {
            listele.Add(tag);
        }
        return listele;
    }
}

这是我的实用工具 class

public class Utilities
{
    public static async Task<string> GetResponseStr(string link)
    {
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(link);
        HttpWebResponse response = (HttpWebResponse)request.GetResponse();

        StreamReader stream = new StreamReader(response.GetResponseStream());
        string final_response = stream.ReadToEnd();
        return final_response;
    }

}

Answer 1

当执行你的请求并获取 HttpWebResponse 时，你可以检查 StatusCode 属性，如果它是 404 或除 200 之外的任何值，那么它可以被视为不可用。

见https://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.statuscode(v=vs.110).aspx

我建议创建一个 class 以在出现任何问题时传递 Web 响应的状态代码

希望对您有所帮助！

Answer 2

您可以像下面这样简单地集成到您的代码中：

public static async Task<string> GetResponseStr(string link)
        {
            var final_response = string.Empty;
            try
            {
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(link);
                HttpWebResponse response = (HttpWebResponse)request.GetResponse();

                StreamReader stream = new StreamReader(response.GetResponseStream());
                final_response = stream.ReadToEnd();
            }
            catch (Exception ex)
            {
                //DO whatever necessary like log or sending email to notify you   
            }

            return final_response;
        }

然后当你调用时，在进一步处理之前添加一个检查：

    htmlStr = Utilities.GetResponseStr(haberYildizi.Link).Result;
    if (!string.IsNullOrEmpty(htmlStr))
    {

    }

Answer 3

我只有端点网址和访问令牌。比我记录的错误请求还多

   if (response.StatusCode == System.Net.HttpStatusCode.OK)
        {
            StreamReader stream = new StreamReader(response.GetResponseStream());
            string final_response = stream.ReadToEnd();
            return final_response;
            }
 else
            {
                Logger.CreateLogEntry("<== WebRequest ", "Could not Connect to server. Server Response Code: " + response.StatusCode);
                //Add bad request handler
                return null;
            }

如何使用 webrequest 检查 link 是否损坏或关闭？

How to check if link is borken or closed with webrequest?

c#

httpwebrequest

httpwebresponse