问题 httpClient.GetStringAsync(URI) 加载

Question

我用这个方法加载有问题。事实上，我想加载一个网页来获取 Html 代码。但是网页没有时间完全加载。所以我想在这个方法中添加一个 thread.sleep() 。你知道我该怎么做吗？

            var html = await httpClient.GetStringAsync(url); 
            HtmlAgilityPack.HtmlDocument htmlDocument = new HtmlAgilityPack.HtmlDocument();
            htmlDocument.LoadHtml(html);

Answer 1

有没有截图:

第一个是用WebClient下载的结果。第二个是网页原Html代码

Answer 2

我和我的老板，我们找到了解决方案。 Selenium 中有一个函数可以从网站获取所有 html 代码。由于 Selenium 在与页面进行任何交互之前完全加载了页面，因此 html 代码被完全加载。这是代码：

driver.Navigate().GoToUrl(url);
driver.Manage().Window.Size = new System.Drawing.Size(1936, 1056);
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);
var result = driver.FindElement(By.TagName("body")).GetAttribute("innerHTML");
await StartCrawlerasync(result);

public static async Task StartCrawlerasync(string html)
        {
            var Links = new List<string>();
            StringBuilder csvcontent = new StringBuilder();
            StringBuilder htmlcontent = new StringBuilder();
            string htmlpath = @"path\Test.html";
            File.WriteAllText(htmlpath, string.Empty);
            File.WriteAllText(htmlpath, html);
            string csvpath = @"path\Tous_les_Liens.csv";
            File.WriteAllText(csvpath, string.Empty);

            var httpClient = new HttpClient();
            HtmlAgilityPack.HtmlDocument htmlDocument = new HtmlAgilityPack.HtmlDocument();
            await Task.Delay(5000);
            htmlDocument.LoadHtml(html);

            if (htmlDocument.DocumentNode.SelectNodes("//a") != null)
            {
                foreach (HtmlNode link in htmlDocument.DocumentNode.SelectNodes("//a"))
                {
                    Links.Add(link.Attributes["href"].Value);
                    csvcontent.AppendLine(link.Attributes["href"].Value);
                };

                foreach (string l in Links)
                {
                    Console.WriteLine(l);
                }
            }
            else
            {
                Console.WriteLine("C''est vide");
            }
            File.WriteAllText(csvpath, csvcontent.ToString());        
        }

问题 httpClient.GetStringAsync(URI) 加载

Problem httpClient.GetStringAsync(URI) load

c#

overwrite

html-agility-pack