使用多个 <table> 标签抓取 HTML 页面并从特定 <a> 标签后代中提取文本
Scrape HTML page with multiple <table> tags and extract text from specific <a> tag descendants
我在数据库字段中有这个 html 源代码。我想分析这段代码,特别是一些 table 的字段,并将它们打印在屏幕上。这是关于 table:
的代码<table cellspacing="1" cellpadding="1" class="troop_details inReturn"
>
<thead>
<tr>
<td class="role">
<a href="/karte.php?d=91628">01] #WorkInProgress</a>
</td>
<td colspan="11" class="troopHeadline">
<a href="/karte.php?d=91611">Return from 01-soldier</a>
</td>
</tr>
</thead>
<tbody class="units">
<tr>
<th class="coords">
‭<span class="coordinates coordinatesWrapper coordinatesAligned coordinatesltr"><span class="coordinateX">(‭−‭1‬‬</span><span class="coordinatePipe">|</span><span class="coordinateY">‭−‭28‬‬)</span></span>‬ </th>
<td class="uniticon">
<img class="unit u21" title="Phalanx: 1:12:51" alt="Phalanx" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u22" title="Swordsman: 1:25:00" alt="Swordsman" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u23" title="Pathfinder: 0:30:00" alt="Pathfinder" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u24" title="Theutates Thunder: 0:26:51" alt="Theutates Thunder" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u25" title="Druidrider: 0:31:53" alt="Druidrider" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u26" title="Haeduan: 0:39:14" alt="Haeduan" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u27" title="Ram: 2:07:30" alt="Ram" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u28" title="Trebuchet: 2:50:00" alt="Trebuchet" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u29" title="Chieftain: 1:42:00" alt="Chieftain" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u30" title="Settler: 1:42:00" alt="Settler" src="/img/x.gif" /> </td>
<td class="uniticon last">
<img class="unit uhero" title="Hero" alt="Hero" src="/img/x.gif" /> </td>
</tr>
</tbody>
<tbody class="units last">
<tr>
<th>Troops</th>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit">
500 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none last">
0 </td>
</tr>
</tbody>
<tbody class="infos">
<tr>
<th>Bounty</th>
<td colspan="11">
<div class="res">
<div class="inlineIconList resourceWrapper"><div class="inlineIcon resources" title="Lumber"><i class="r1"></i><span class="value ">6758</span></div><div class="inlineIcon resources" title="Clay"><i class="r2"></i><span class="value ">8093</span></div><div class="inlineIcon resources" title="Iron"><i class="r3"></i><span class="value ">6908</span></div><div class="inlineIcon resources" title="Crop"><i class="r4"></i><span class="value ">15741</span></div></div> </div>
<div class="carry">
<img class="carry full" title="carry"
alt="carry"
src="/img/x.gif"/> ‭‭37500‬ / ‭37500‬‬ </div>
</td>
</tr>
</tbody>
<tbody class="infos">
<tr>
<th>Arrival</th>
<td colspan="11">
<div class="in">in <span class="timer" counting="down" value="85">0:01:25</span> hrs.</div>
<div class="at"><span>at 00:43:10</span><span> </span></div>
</td>
</tr>
</tbody>
</table>
<a name="at"></a>
<table cellspacing="1" cellpadding="1" class="troop_details inReturn"
>
<thead>
<tr>
<td class="role">
<a href="/karte.php?d=91628">01] #WorkInProgress</a>
</td>
<td colspan="11" class="troopHeadline">
<a href="/karte.php?d=94829">Return from 0-New Hulk</a>
</td>
</tr>
</thead>
<tbody class="units">
<tr>
<th class="coords">
‭<span class="coordinates coordinatesWrapper coordinatesAligned coordinatesltr"><span class="coordinateX">(‭−‭1‬‬</span><span class="coordinatePipe">|</span><span class="coordinateY">‭−‭28‬‬)</span></span>‬ </th>
<td class="uniticon">
<img class="unit u21" title="Phalanx: 0:45:33" alt="Phalanx" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u22" title="Swordsman: 0:53:09" alt="Swordsman" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u23" title="Pathfinder: 0:18:46" alt="Pathfinder" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u24" title="Theutates Thunder: 0:16:47" alt="Theutates Thunder" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u25" title="Druidrider: 0:19:56" alt="Druidrider" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u26" title="Haeduan: 0:24:32" alt="Haeduan" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u27" title="Ram: 1:19:44" alt="Ram" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u28" title="Trebuchet: 1:46:18" alt="Trebuchet" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u29" title="Chieftain: 1:03:47" alt="Chieftain" src="/img/x.gif" /> </td>
<td class="uniticon">
<img class="unit u30" title="Settler: 1:03:47" alt="Settler" src="/img/x.gif" /> </td>
<td class="uniticon last">
<img class="unit uhero" title="Hero" alt="Hero" src="/img/x.gif" /> </td>
</tr>
</tbody>
<tbody class="units last">
<tr>
<th>Troops</th>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit">
400 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none">
0 </td>
<td class="unit none last">
0 </td>
</tr>
</tbody>
<tbody class="infos">
<tr>
<th>Bounty</th>
<td colspan="11">
<div class="res">
<div class="inlineIconList resourceWrapper"><div class="inlineIcon resources" title="Lumber"><i class="r1"></i><span class="value ">6130</span></div><div class="inlineIcon resources" title="Clay"><i class="r2"></i><span class="value ">5835</span></div><div class="inlineIcon resources" title="Iron"><i class="r3"></i><span class="value ">5638</span></div><div class="inlineIcon resources" title="Crop"><i class="r4"></i><span class="value ">12397</span></div></div> </div>
<div class="carry">
<img class="carry full" title="carry"
alt="carry"
src="/img/x.gif"/> ‭‭30000‬ / ‭30000‬‬ </div>
</td>
</tr>
</tbody>
<tbody class="infos">
<tr>
<th>Arrival</th>
<td colspan="11">
<div class="in">in <span class="timer" counting="down" value="920">0:15:20</span> hrs.</div>
<div class="at"><span>at 00:57:05</span><span> </span></div>
</td>
</tr>
</tbody>
</table>
我感兴趣的数据如下:
- Return 来自 01-士兵 00:43:10
- Return 来自 0-新绿巨人 00:57:05
感谢您的建议,这是我目前的代码:
<?php include 'database.php' ?>
<?php session_start(); ?>
<?php
include_once('simple_html_dom.php');
$caserma = $_SESSION["caserma"];
$dom = new DOMDocument;
libxml_use_internal_errors(true);
$dom->loadHTML($_SESSION["caserma"], LIBXML_HTML_NOIMPLIED | LIBXML_HTML_NODEFDTD);
$xpath = new DOMXPath($dom);
$texts = [];
foreach ($xpath->query("//table[contains(@class, 'troop_details') and contains(@class, 'inReturn')]//td[@class='troopHeadline']//a[@href]/text()") as $textNode) {
$texts[] = $textNode->nodeValue;
}
var_export($texts);
?>
但是作为输出它给了我数组 ( )
代码假定 $_SESSION["caserma"]
包含完整的 html 文档:(Demo)
$dom = new DOMDocument;
libxml_use_internal_errors(true);
$dom->loadHTML($_SESSION["caserma"]);
$xpath = new DOMXPath($dom);
$texts = [];
foreach ($xpath->query("//table[contains(@class, 'troop_details') and contains(@class, 'inReturn')]//td[@class='troopHeadline']//a[@href]/text()") as $textNode) {
$texts[] = $textNode->nodeValue;
}
var_export($texts);
样本输入的输出:
array (
0 => 'Return from 01-soldier',
1 => 'Return from 0-New Hulk',
)
XPath 分解:
// # search to any depth in the document
table[contains(@class, 'troop_details') and contains(@class, 'inReturn')] # find all table tags with both `troop_details` and `inReturn` classes
// # continue searching any descendants of any matches
td[@class='troopHeadline'] # match all td tags with `troopHeadline` as its class
// # continue searching anydescendants of any matches
a[@href] # match all a tags with an href attribute
/ # search the immediate descendant (any first generation child)
text() # match the text of the parent a tag
libxml_use_internal_errors(true)
用于消除来自“无效”文档的任何潜在错误。- 在 xpath 中使用
contains(...) and contains()
很重要,这样即使 class 属性改变了它们的顺序或新的 classes 添加到元素,xpath 仍然会正确匹配。 foreach()
循环将迭代所有符合条件的文本节点。- 提取
nodeValue
并将其推入结果数组。