Use MultiDict for headers about urllib3 HOT 4 CLOSED

urllib3 commented on August 24, 2024

Use MultiDict for headers

from urllib3.

Comments (4)

piotr-dobrogost commented on August 24, 2024

Currently httplib coerces headers into a dict, which breaks some things

Well, to be precise the problem is not dict itself but merging headers with the same name into one entry.
It's worth noting it was fixed in Python 3:

2.7.2

>>> import httplib
>>> conn = httplib.HTTPConnection('www.wp.pl')
>>> conn.request('HEAD', '/index.html')
>>> res = conn.getresponse()
>>> print res.getheaders()
[('set-cookie', 'reksticket=1323027255; expires=Tuesday, 06-Dec-11 19:34:15 GMT; path=/; domain=.www.wp.pl,
rekticket=1323027255; expires=Tuesday, 06-Dec-11 19:34:15 GMT; path=/; domain=.wp.pl,
statid=77.255.32.195.20351:1323027255:2355453207:v1; path=/; expires=Wed, 03-Dec-14 19:34:15 GMT,
statid=77.255.32.195.20351:1323027255:2355453207:v1; domain=.wp.pl; path=/; expires=Wed, 03-Dec-14 19:34:15 GMT'),...]

3.2.2

>>> from http import client
>>> conn = client.HTTPConnection('www.wp.pl')
>>> conn.request('HEAD', '/index.html')
>>> res = conn.getresponse()
>>> print(res.getheaders())
[('Set-Cookie', 'reksticket=1323028283; expires=Tuesday, 06-Dec-11 19:51:23 GMT; path=/; domain=.www.wp.pl'), ('Set-Cookie',
'rekticket=1323028283; expires=Tuesday, 06-Dec-11 19:51:23 GMT; path=/; domain=.wp.pl'), ('Set-Cookie',
'statid=77.255.32.195.7252:1323028283:2993994873:v1; path=/; expires=Wed, 03-Dec-14 19:51:23 GMT'), ('Set-Cookie',
'statid=77.255.32.195.7252:1323028283:2993994873:v1; domain=.wp.pl; path=/; expires=Wed, 03-Dec-14 19:51:23 GMT'), ...]

This functionality is provided by (undocumented?) HTTPMessage class and Python issue 4773 HTTPMessage not documented and has inconsistent API across 2.6/3.0 is of interest here. The difference shown above is noted by Joel Verhagen in comment http://bugs.python.org/issue4773#msg154781.

The issue we see was raised as issue 1660009 continuing problem with httplib multiple set-cookie headers Python issue.
John J Lee suggests using getallmatchingheaders() method or headers property of rfc822.Message class to get raw header data.

from urllib3.