如何在 python 中获取带有 unicode 字符的网页
How to get a webpage with unicode chars in python
我正在尝试获取并解析包含非 ASCII 字符的网页（URL 是 http://www.one.co.il）。这是我的：
url = "http://www.one.co.il"
req = urllib2.Request(url)
response = urllib2.urlopen(req)
encoding = response.headers.getparam('charset') # windows-1255
html = response.read() # The length of this is valid - about 31000-32000,
# but printing the first characters shows garbage -
# '\x1f\x8b\x08\x00\x00\x00\x00\x00', instead of
# '<!DOCTYPE'
html_decoded = html.decode(encoding)
最后一行给了我一个例外：
File "C:/Users/....\WebGetter.py", line 16, in get_page
html_decoded = html.decode(encoding)
File "C:\Python27\lib\encodings\cp1255.py", line 15, in decode
return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0xdb in position 14: character maps to <undefined>
我尝试查看其他相关问题，例如 urllib2 read to Unicode and How to handle response encoding from urllib.request.urlopen()，但没有找到任何有用的信息。
有人可以在这个主题上阐明并指导我吗？谢谢！
0x1f 0x8b 0x08 是压缩文件的幻数。需要先解压才能使用。
我正在尝试获取并解析包含非 ASCII 字符的网页（URL 是 http://www.one.co.il）。这是我的：
url = "http://www.one.co.il"
req = urllib2.Request(url)
response = urllib2.urlopen(req)
encoding = response.headers.getparam('charset') # windows-1255
html = response.read() # The length of this is valid - about 31000-32000,
# but printing the first characters shows garbage -
# '\x1f\x8b\x08\x00\x00\x00\x00\x00', instead of
# '<!DOCTYPE'
html_decoded = html.decode(encoding)
最后一行给了我一个例外：
File "C:/Users/....\WebGetter.py", line 16, in get_page
html_decoded = html.decode(encoding)
File "C:\Python27\lib\encodings\cp1255.py", line 15, in decode
return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0xdb in position 14: character maps to <undefined>
我尝试查看其他相关问题，例如 urllib2 read to Unicode and How to handle response encoding from urllib.request.urlopen()，但没有找到任何有用的信息。
有人可以在这个主题上阐明并指导我吗？谢谢！
0x1f 0x8b 0x08 是压缩文件的幻数。需要先解压才能使用。