如何使用 mechanize 和 nokogiri 获取链接 ruby
How to get links using mechanize and nokogiri ruby
鉴于下面的示例,任何人都可以告诉我如何使用 Nokogiri 和 Mechanize 获取单独组中每个 <h4>
标签下的所有链接,I.E.以下所有链接:
- "some text"
- "some more text"
- "some additional text"
<div id="right_holder">
<h3><a href="#"><img src="http://example.com" width="11" height="11"></a></h3>
<br />
<br />
<h4><a href="#">Some text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<br />
<br />
<h4><a href="#">Some more text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<br />
<br />
<h4><a href="#">Some additional text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
</div>
您可以像“How to split a HTML document using Nokogiri?”一样遍历并分隔数据,但如果您知道标签是什么,您可以 split
它:
# html is the raw html string
html.split('<h4').map{|g| Nokogiri::HTML::DocumentFragment.parse(g).css('a') }
page = Nokogiri::HTML(html).css("#right_holder")
links = page.children.inject([]) do |link_hash, child|
if child.name == 'h4'
name = child.text
link_hash << { :name => name, :content => ""}
end
next link_hash if link_hash.empty?
link_hash.last[:content] << child.to_xhtml
link_hash
end
grouped_hsh = links.inject({}) do |hsh, link|
hsh[link[:name]] = Nokogiri::HTML::DocumentFragment.parse(link[:content]).css('a')
hsh
end
# {"Some text"=>[#<Nokogiri::XML::Element:0x3ff4860d6c30,
# "Some more text"=>[#<Nokogiri::XML::Element:0x3ff486096c20...,
# "Some additional text"=>[#<Nokogiri::XML::Element:0x3ff486f2de78...}
一般来说你会这样做:
page.search('h4 a').each do |a|
puts a[:href]
end
但我相信您已经注意到 none 这些链接实际上可以到达任何地方。
更新:
要对它们进行分组,一些节点集数学如何:
page.search('h4').each do |h4|
puts h4.text
(h4.search('~ a') - h4.search('~ h4 ~ a')).each do |a|
puts a.text
end
end
这意味着每个 a
都跟在 h4
之后并且不跟在另一个 h4
之后
鉴于下面的示例,任何人都可以告诉我如何使用 Nokogiri 和 Mechanize 获取单独组中每个 <h4>
标签下的所有链接,I.E.以下所有链接:
- "some text"
- "some more text"
- "some additional text"
<div id="right_holder">
<h3><a href="#"><img src="http://example.com" width="11" height="11"></a></h3>
<br />
<br />
<h4><a href="#">Some text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<br />
<br />
<h4><a href="#">Some more text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<br />
<br />
<h4><a href="#">Some additional text</a></h4>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
<a href="#" alt="name of item"><img src="http://some.image.com" class="class1"></a>
</div>
您可以像“How to split a HTML document using Nokogiri?”一样遍历并分隔数据,但如果您知道标签是什么,您可以 split
它:
# html is the raw html string
html.split('<h4').map{|g| Nokogiri::HTML::DocumentFragment.parse(g).css('a') }
page = Nokogiri::HTML(html).css("#right_holder")
links = page.children.inject([]) do |link_hash, child|
if child.name == 'h4'
name = child.text
link_hash << { :name => name, :content => ""}
end
next link_hash if link_hash.empty?
link_hash.last[:content] << child.to_xhtml
link_hash
end
grouped_hsh = links.inject({}) do |hsh, link|
hsh[link[:name]] = Nokogiri::HTML::DocumentFragment.parse(link[:content]).css('a')
hsh
end
# {"Some text"=>[#<Nokogiri::XML::Element:0x3ff4860d6c30,
# "Some more text"=>[#<Nokogiri::XML::Element:0x3ff486096c20...,
# "Some additional text"=>[#<Nokogiri::XML::Element:0x3ff486f2de78...}
一般来说你会这样做:
page.search('h4 a').each do |a|
puts a[:href]
end
但我相信您已经注意到 none 这些链接实际上可以到达任何地方。
更新:
要对它们进行分组,一些节点集数学如何:
page.search('h4').each do |h4|
puts h4.text
(h4.search('~ a') - h4.search('~ h4 ~ a')).each do |a|
puts a.text
end
end
这意味着每个 a
都跟在 h4
之后并且不跟在另一个 h4