Python，UTF-8 URI 解码

Question

Python 新手（当然）在解码 URI 编码的字符串时遇到一些问题。

我的代码：

#/usr/bin/env python
# -*- coding: utf-8 -*-
# Encoding: UTF-8
...
import urllib
...
secondTag = urllib.unquote(secondTag).decode('utf8')
...

什么时候

secondTag = "flashvars=%7B%22video%22%3A%7B%22videoReferences%22%3A%5B%7B%22url%22%3A%22http%3A%2F%2Fsvtplay6a-f.akamaihd.net%2Fz%2Fse%2Fopen%2F20160405%2F1114066-002A%2FPG-1114066-002A-AFFARENRAMEL-01_%2C988%2C240%2C348%2C456%2C636%2C1680%2C2796%2C.mp4.csmil%2Fmanifest.f4m%22%2C%22playerType%22%3A%22flash%22%7D%5D%2C%22subtitleReferences%22%3A%5B%7B%22url%22%3A%22http%3A%2F%2Fmedia.svt.se%2Fdownload%2Fmcc%2Ftest%2Fcore-prd%2FSUB-1114066-002A-AFFARENRAMEL%2FSUB-1114066-002A-AFFARENRAMEL.wsrt%22%7D%5D%2C%22position%22%3A0%7D%2C%22statistics%22%3A%7B%22client%22%3A%22nojs%22%2C%22mmsClientNr%22%3A%221001001%22%2C%22programId%22%3A%221114066-002A%22%2C%22statisticsUrl%22%3A%22%2F%2Fld.svt.se%2Fsvt%2Fsvt%2Fs%3Fnojs.Aff%C3%A4ren%20Ramel.Avsnitt%202%22%2C%22title%22%3A%22Avsnitt%202%22%2C%22folderStructure%22%3A%22Aff%C3%A4ren%20Ramel%22%7D%2C%22context%22%3A%7B%7D%7D"

结果是：

File "/home/mythtv/bin/pyPirateDownloader/svtPlay.py", line 70, in checkSecondSvtPage
secondTag = urllib.unquote(secondTag).decode('utf8')
File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 509-510: ordinal not in range(128)

然而，当我在 python 控制台中运行相同时，我得到了预期的结果：

>>> import urllib
>>> secondTag = "flashvars=%7B%22video%22%3A%7B%22videoReferences%22%3A%5B%7B%22url%22%3A%22http%3A%2F%2Fsvtplay6a-f.akamaihd.net%2Fz%2Fse%2Fopen%2F20160405%2F1114066-002A%2FPG-1114066-002A-AFFARENRAMEL-01_%2C988%2C240%2C348%2C456%2C636%2C1680%2C2796%2C.mp4.csmil%2Fmanifest.f4m%22%2C%22playerType%22%3A%22flash%22%7D%5D%2C%22subtitleReferences%22%3A%5B%7B%22url%22%3A%22http%3A%2F%2Fmedia.svt.se%2Fdownload%2Fmcc%2Ftest%2Fcore-prd%2FSUB-1114066-002A-AFFARENRAMEL%2FSUB-1114066-002A-AFFARENRAMEL.wsrt%22%7D%5D%2C%22position%22%3A0%7D%2C%22statistics%22%3A%7B%22client%22%3A%22nojs%22%2C%22mmsClientNr%22%3A%221001001%22%2C%22programId%22%3A%221114066-002A%22%2C%22statisticsUrl%22%3A%22%2F%2Fld.svt.se%2Fsvt%2Fsvt%2Fs%3Fnojs.Aff%C3%A4ren%20Ramel.Avsnitt%202%22%2C%22title%22%3A%22Avsnitt%202%22%2C%22folderStructure%22%3A%22Aff%C3%A4ren%20Ramel%22%7D%2C%22context%22%3A%7B%7D%7D"
>>> secondTag = urllib.unquote(secondTag).decode('utf8')
>>> print secondTag
flashvars={"video":{"videoReferences":[{"url":"http://svtplay6a-f.akamaihd.net/z/se/open/20160405/1114066-002A/PG-1114066-002A-AFFARENRAMEL-01_,988,240,348,456,636,1680,2796,.mp4.csmil/manifest.f4m","playerType":"flash"}],"subtitleReferences":[{"url":"http://media.svt.se/download/mcc/test/core-prd/SUB-1114066-002A-AFFARENRAMEL/SUB-1114066-002A-AFFARENRAMEL.wsrt"}],"position":0},"statistics":{"client":"nojs","mmsClientNr":"1001001","programId":"1114066-002A","statisticsUrl":"//ld.svt.se/svt/svt/s?nojs.Affären Ramel.Avsnitt 2","title":"Avsnitt 2","folderStructure":"Affären Ramel"},"context":{}}

当然这是一些编码问题，我猜这与'ä'字符有关，因为当不存在瑞典字符时不会出现此问题，但我只是不知道为什么和如何解决。

有人能够解释并可能帮助解决这个问题吗？

谢谢 /乔恩

Answer 1

注意是Unicode编码错误：这不是解码失败，这是编码失败！

因为 Python 2 会自动在 str 和 unicode 之间转换，如果您尝试解码 unicode 字符串，可能会遇到编码错误。

这很可能是这里发生的事情：我假设在文件中，secondTag 是一个 unicode 对象：urllib.unquote 然后 return 一个 unicode 对象，所以当你尝试解码它时，它首先尝试将它编码为 str 对象，以便它可以解码它，使用默认的 ascii 编码，但失败了。

没有特别优雅的方式来处理这个问题。可能最优雅的方式是urllib.unquote(secondTag.encode('utf8')).decode('utf8')。如果你想处理它已经是 str 的情况，你可以很容易地添加 if isinstance(secondTag, unicode) else secondTag.

Python，UTF-8 URI 解码

Python, UTF-8 URI-decoding

python

utf-8

urldecode