使用漂亮的汤，在具有 class 的 html 元素中仅刮取文本

Question

我正在尝试使用 BeatifulSoup 抓取页面

import urllib2
from bs4 import BeautifulSoup

url='http://www.xpn.org/playlists/xpn-playlist'
page = urllib2.urlopen(url)

soup = BeautifulSoup(page.read())

for link in soup.find_all("li", class_="song"):
    print link

问题是我想要 return 的文本没有包含在它自己的 html 标签中

<li class="song"> <a href="/default.htm" onclick="return clickreturnvalue()
" onmouseout="delayhidemenu()" onmouseover="dropdownmenu(this, event, menu1, 
'100px','Death Vessel','Mandan Dink','Stay Close')">Buy</a>  
Chuck Ragan - Rotterdam - Folkadelphia Session</li>

我想要什么return Chuck Ragan - Rotterdam - Folkadelphia Session

奖励积分：数据returned 的格式为Artist/Song/Album。用于存储和操作此信息的正确数据结构是什么？

Answer 1

试试这样的东西：

for link in soup.find_all("li", class_="song"):
    print link.text

输出：

Buy  Chuck Ragan - Rotterdam - Folkadelphia Session

当然，如果你想删除Buy，你可以像这样使用slice：

for link in soup.find_all("li", class_="song"):
    print link.text.strip()[5:]

输出为：

Chuck Ragan - Rotterdam - Folkadelphia Session

如果您想将这些字符串保存在列表中：

[i.strip() for i in link.text.strip()[5:].split('-')]

输出：

['Chuck Ragan', 'Rotterdam', 'Folkadelphia Session']

更多信息，您可以查看document。

Answer 2

你可以使用这样的东西。

for l in soup.find_all("li", class_="song"):
    album = l.text.split("-")[2]
    song = l.text.split("-")[1]
    artist = l.text.split("-")[0].split(" ")[1]

Answer 3

这是另一种方式！（假设li有3个children。如果没有，将[2]改为[1]）：

>>> html = '''<li class="song"> <a href="/default.htm" onclick="return clickreturnvalue()
... " onmouseout="delayhidemenu()" onmouseover="dropdownmenu(this, event, menu1,
... '100px','Death Vessel','Mandan Dink','Stay Close')">Buy</a>
... Chuck Ragan - Rotterdam - Folkadelphia Session</li>'''

>>> from bs4 import BeautifulSoup as bs
>>> all_li = soup.findAll('li', class_='song')
>>> for li in all_li:
...     text = list(li.children)[2]
...     artist, song, album = text.split('-')
...     print artist, song, album
Chuck Ragan   Rotterdam   Folkadelphia Session

Answer 4

** 最终使用 named tuple 进行存储 **

from bs4 import BeautifulSoup
import urllib2
from collections import namedtuple

url='http://www.xpn.org/playlists/xpn-playlist'
page = urllib2.urlopen(url)


soup = BeautifulSoup(page.read())

songs=[]
Song = namedtuple("Song", "artist name album")
for link in soup.find_all("li", class_="song"):
    song = Song._make(link.text.strip()[12:].split(" - "))
    songs.append(song)

for song in songs:
    print(song.artist, song.name, song.album)

使用漂亮的汤，在具有 class 的 html 元素中仅刮取文本

Scrape just the text, within an html element that has a class, using beautiful soup

python

beautifulsoup