我的代码使用urlib / httplib发布文档,如下所示：
request = urllib2.Request(self.url,xml_request.toxml('utf-8') if xml_request else None,self.headers)
obj = parse_xml(self.opener.open(request))
当XML文本包含非ASCII字符时,这会产生异常：
File "/usr/lib/python2.7/httplib.py",line 951,in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py",line 809,in _send_output msg += message_body exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 89: ordinal not in range(128)
就像我所看到的那样,httplib._send_output正在为消息有效负载创建一个ASCII字符串,大概是因为它希望数据是URL编码的(application / x-www-form-urlencoded).只要使用ASCII字符,它就可以与application / xml一起使用.
是否有一种直接的方式来发布包含非ASCII字符的应用程序/ xml数据,或者我将不得不跳过箍(例如使用Twistd和POST有效负载的自定义生产者)？
解决方法
>>> msg = u'abc' # Unicode string >>> message_body = b'\xc5' # bytestring >>> msg += message_body Traceback (most recent call last): File "<input>",line 1,in <module> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 0: ordinal \ not in range(128)
要修复它,请确保self.headers内容已正确编码,即标题中的所有键,值应为bytestrings：
self.headers = dict((k.encode('ascii') if isinstance(k,unicode) else k,v.encode('ascii') if isinstance(v,unicode) else v)
for k,v in self.headers.items())
注意：标题的字符编码与正文的字符编码无关,即xml文本可以独立编码(它只是来自http消息的八位字节流).
self.url也是如此 – 如果它具有unicode类型;将其转换为bytestring(使用’ascii’字符编码).
HTTP message consists of a start-line,“headers”,an empty line and possibly a message-body所以self.headers用于头文件,self.url用于启动线(http方法在这里),可能用于Host http头文件(如果客户端是http / 1.1),XML文本转到消息体(作为二进制blob) ).
对self.url使用ASCII编码总是安全的(IDNA可用于非ascii域名 – 结果也是ASCII).
这是rfc 7230 says about http headers character encoding：
Historically,HTTP has allowed field content with text in the
ISO-8859-1 charset [ISO-8859-1],supporting other charsets only
through use of [RFC2047] encoding. In practice,most HTTP header
field values use only a subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit their field values to
US-ASCII octets. A recipient SHOULD treat other octets in field
content (obs-text) as opaque data.
要将XML转换为字节字符串,请参阅application/xml encoding condsiderations：
The use of UTF-8,without a BOM,is RECOMMENDED for all XML MIME entities.