Php 使用简单 html dom 的网络抓取在输出乱序时无法正常工作 html 标签

Question

我想抓取网页的一些信息。它使用 table 布局结构。

我想在包含一系列嵌套 table 的嵌套 table 布局中提取第三个 table。每个都发布一个结果。但是代码不起作用

include('simple_html_dom.php');
$url = 'http://exams.keralauniversity.ac.in/Login/index.php?reslt=1';
$html = file_get_contents($url);
$result =$html->find("table", 2);
echo $result;

我使用 Curl 提取网站，但问题是它的标签乱序，因此无法使用简单的 dom 元素提取。

    function curl($url) {
            $ch = curl_init();  // Initialising cURL
            curl_setopt($ch, CURLOPT_URL,$url);    // Setting cURL's URL option with the $url variable passed into the function
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
            $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
            curl_close($ch);    // Closing cURL
            return $data;   // Returning the data from the function
        }

          function scrape_between($data, $start, $end){
        $data = stristr($data, $start); // Stripping all data from before $start
        $data = substr($data, strlen($start));  // Stripping $start
        $stop = stripos($data, $end);   // Getting the position of the $end of the data to scrape
        $data = substr($data, 0, $stop);    // Stripping all data from after and including the $end of the data to scrape
        return $data;   // Returning the scraped data from the function
    }
          $scraped_page  = curl($url);  // Executing our curl function to scrape the webpage http://www.example.com and return the results into the $scraped_website variable

           $scraped_data = scrape_between($scraped_page, ' </html>', '</table></td><td></td></tr>
   </table>');  
 echo $scraped_data;
 $myfile = fopen("newfile.html", "w") or die("Unable to open file!");

fwrite($myfile, $scraped_data);
fclose($myfile);

如何抓取结果并保存 pdf

Answer 1

Find a sample code


    ?php
        // Defining the basic cURL function
        function curl($url) {
            $ch = curl_init();  // Initialising cURL
            curl_setopt($ch, CURLOPT_URL, $url);    // Setting cURL's URL option with the $url variable passed into the function
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Setting cURL's option to return the webpage data
            $data = curl_exec($ch); // Executing the cURL request and assigning the returned data to the $data variable
            curl_close($ch);    // Closing cURL
            return $data;   // Returning the data from the function
        }
    ?>

    <?php
        $scraped_website = curl("http://www.example.com");  // Executing our curl function to scrape the webpage http://www.example.com and return the results into the $scraped_website variable
$result =$substring($scraped_website ,11,7); //change values 11,7 for table
echo $result;
    ?>

Answer 2

简单 HTML Dom 无法处理 html。所以先切换到this library，然后做：

require_once('advanced_html_dom.php');

$dom = file_get_html('http://exams.keralauniversity.ac.in/Login/index.php?reslt=1');

$rows = array();
foreach($dom->find('tr.Function_Text_Normal:has(td[3])') as $tr){
  $row['num'] = $tr->find('td[2]', 0)->text;
  $row['text'] = $tr->find('td[3]', 0)->text;
  $row['pdf'] = $tr->find('td[3] a', 0)->href;
  if(preg_match_all('/\d+/', $tr->parent->find('u', 0)->text, $m)){
    list($row['day'], $row['month'], $row['year']) = $m[0];
  }

  // uncomment next 2 lines to save the pdf
  // $filename = preg_replace('/.*\//', '', $row['pdf']);
  // file_put_contents($filename, file_get_contents($row['pdf']));
  $rows[] = $row;
}
var_dump($rows);

Php 使用简单 html dom 的网络抓取在输出乱序时无法正常工作 html 标签

Php webscraping using simple html dom not working when output is out of order html tags

php

simple-html-dom

web-scraping