当我使用 urllib2 抓取网站时，但没有标签，例如 html,body

Question

import urllib2

url = 'http://www.bilibili.com/video/av1669338'

user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"

headers={"User-Agent":user_agent}

request=urllib2.Request(url,headers=headers)

response=urllib2.urlopen(request)

text = response.read()

text[:100]

'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03\xcd}ys\x1bG\xb2\xe7\xdfV\xc4|\x87\x1exhRk\x81\xb8\x08\x10\x90E\xfa\x89\xb2f\x9f\xe3\xd9\xcf\x9e\x1dyb7\xec\tD\x03h\x90\x90p\t\x07)yf"D\xf9I&EI\xd4}\x91\xb6.\xeb\xb0e\x93\x94% Y\xbc$E\xccW\x194\x00\xfe\xe5\xaf\xf0~Y\xd5\xd5\xa8\xeeF\x83\xa7'

Answer 1

试试这个：

import bs4, requests
res = requests.get("http://www.bilibili.com/video/av1669338")
soup = bs4.BeautifulSoup(res.content, "lxml")
result = soup.find("meta", attrs = {"name":"keywords"}).get("content")
print result

Answer 2

导入请求从 bs4 导入 BeautifulSoup

定义数据（）： url = 'http://www.bilibili.com/video/av1669338' user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36" headers = {"User-Agent": user_agent} 响应 = requests.get(url, headers=headers)

data = response.content
_html = BeautifulSoup(data)
_meta = _html.head.select('meta[name=keywords]')
print _meta[0]['content']

当我使用 urllib2 抓取网站时，但没有标签，例如 html,body

when I use urllib2 to crawl a wibsite,but without labels ,such as html,body

python

urllib2

labels