Code Monkey home page Code Monkey logo

nectar's Introduction

⚠️ ⛔️ Pulp2 is EOL as of November 30 2022, for more info visit this link https://pulpproject.org/2022/09/19/pulp-2-eol/. ⛔️

Pulp is a platform for managing repositories of content, such as software
packages, and pushing that content out to large numbers of consumers.

For more information, check out the project website:

http://www.pulpproject.org

nectar's People

Contributors

alexjfisher avatar ammaritiz avatar asmacdo avatar barnabycourt avatar daviddavis avatar dkliban avatar evgeni avatar ipanova avatar jeremycline avatar jlconnor avatar jortel avatar mhrivnak avatar midnightercz avatar pcreech avatar sean797 avatar twaugh avatar werwty avatar yhekma avatar zjhuntin avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

dhajoshi lsjostro ryhennessy bowlofeggs asmacdo beav pombreda ipanova osabina pcreech jeremycline shubham90 bmbouter seandst frenzymadness pronix goosemania daviddavis alexjfisher werwty zjhuntin ammaritiz dkliban twaugh midnightercz atix-ag jjansky1 amcmahon-rh

nectar's Issues

UnicodeEncodeError raised on connecting to a website with latin-1 characters in credentials

the only way on how to sync an HTTP-authed repo in katello is to provide the credentials in the URL in a form of:
http://username:[email protected].

This is causing nectar to fail if any of the username or password contains unicode characters.
e.g.:

# hammer --csv repository create --name foo --product-id 1 --organization-id 1 --content-type yum --url "http://%F0%A0%BB%8E%F0%AB%84%B7%F0%A7%B0%A8%E4%97%B2%E6%96%BA%F0%A2%A4%B8%F0%A1%99%8F%E3%9E%8C%ED%9B%BA%EC%A9%A5:%F0%A4%85%AF%F0%A5%91%A2%F0%A2%B3%A4%EA%B5%A9%F0%90%AC%84%F0%A6%B3%94%EC%85%BE%F0%A6%B6%8D%EC%9A%B3%F0%A4%99%[email protected]/fakerepo01/"
# ..i know, what you're thinking :)

# hammer -u admin -p changeme repository synchronize --id 1
[..............................................................................................................] [100%]
No new packages.
Error: RPM1004: Error retrieving metadata: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)

syslog traceback:

May 18 12:04:25 foo.bar pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
May 18 12:04:25 foo.bar pulp: kombu.transport.qpid:INFO: Connected to qpid with SASL mechanism ANONYMOUS
May 18 12:04:25 foo.bar pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[8cdde2a4-d21a-4faf-8d33-74949dbba830]
May 18 12:04:25 foo.bar pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[fff399b3-803a-40c5-bb03-0430b56f8f84]
May 18 12:04:25 foo.bar pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[9d594c8f-4fc5-4ae9-a707-b11b94d1f1ac]
May 18 12:04:25 foo.bar pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://rplevka.fedorapeople.org/fakerepo01/.
May 18 12:04:25 foo.bar pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[8cdde2a4-d21a-4faf-8d33-74949dbba830] succeeded in 0.0346152009997s: None
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) Traceback (most recent call last):
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.config.read_timeout))
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     return self.request('GET', url, **kwargs)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     prep = self.prepare_request(req)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.prepare_auth(auth, url)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     r = auth(self)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:25 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) Traceback (most recent call last):
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     self.config.read_timeout))
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     return self.request('GET', url, **kwargs)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     prep = self.prepare_request(req)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     self.prepare_auth(auth, url)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     r = auth(self)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:26 foo.bar pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://rplevka.fedorapeople.org/fakerepo01/.
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) Traceback (most recent call last):
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.config.read_timeout))
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     return self.request('GET', url, **kwargs)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     prep = self.prepare_request(req)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.prepare_auth(auth, url)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     r = auth(self)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:26 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) Traceback (most recent call last):
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     self.config.read_timeout))
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     return self.request('GET', url, **kwargs)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     prep = self.prepare_request(req)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     self.prepare_auth(auth, url)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     r = auth(self)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-74752) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://rplevka.fedorapeople.org/fakerepo01/.
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) Traceback (most recent call last):
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.config.read_timeout))
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     return self.request('GET', url, **kwargs)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     prep = self.prepare_request(req)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     self.prepare_auth(auth, url)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     r = auth(self)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-46784) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) Traceback (most recent call last):
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     self.config.read_timeout))
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     return self.request('GET', url, **kwargs)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     prep = self.prepare_request(req)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     self.prepare_auth(auth, url)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     r = auth(self)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:27 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) Traceback (most recent call last):
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 221, in _fetch
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     self.config.read_timeout))
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 476, in get
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     return self.request('GET', url, **kwargs)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 450, in request
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     prep = self.prepare_request(req)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 381, in prepare_request
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     hooks=merge_hooks(request.hooks, self.hooks),
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 308, in prepare
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     self.prepare_auth(auth, url)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/models.py", line 495, in prepare_auth
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     r = auth(self)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/nectar/config.py", line 243, in __call__
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     req.headers['Authorization'] = requests.auth._basic_auth_str(self.username, self.password)
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)   File "/usr/lib/python2.6/site-packages/requests/auth.py", line 30, in _basic_auth_str
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752)     b64encode(('%%s:%%s' %% (username, password)).encode('latin1')).strip()
May 18 12:04:28 foo.bar pulp: nectar.downloaders.threaded:ERROR: (26494-98752) UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)
May 18 12:04:29 foo.bar pulp: pulp.server.async.tasks:INFO: Task failed : [fff399b3-803a-40c5-bb03-0430b56f8f84] : Error retrieving metadata: 'latin-1' codec can't encode characters in position 0-9: ordinal not in range(256)

Add the ability to explicitly unset proxy_url in inherited configs

I am running Katello and for nearly all of my repos I want to have a proxy configured. However I am mirroring some repos from internal sources. The repository config seems to inherit from nectar defaults so you can actually override proxy_url on specific repos. But setting proxy_url to None, or "" doesn't work as expected. Because config.proxy_url=""==False in python and when I tried "None" it correctly becomes the string "None" and there is no proxy server named "None" in our network.

I found a workaround, a gross workaround, I manually edited the installed downloaders/threaded.py.

--- theirs/threaded.py  2017-10-12 00:48:19.658019171 +0000
+++ mine/threaded.py    2017-10-12 00:48:37.947019171 +0000
@@ -149,7 +149,7 @@
         #
         # Annoyingly, although the config is called 'proxy_url', the port and basic auth
         # credentials are defined separately, so we have to build the url.
-        if config.proxy_url and config.proxy_port:
+        if config.proxy_url and config.proxy_url != '_none_' and config.proxy_port:
             parsed_url = urllib3_url.parse_url(config.proxy_url)
             proxy_auth = None
             if config.proxy_username and config.proxy_password:

I would submit it as a merge request but I suspect my patch is a bit myopic.

I chose the value _none_ because that's what yum uses and it made sense to me and might make sense to others. I'm not married to it. Just a thought.

Please, return HTTPCurlDownloader and HTTPSCurlDownloader

HTTPCurlDownloader and HTTPSCurlDownloader - pycurl module.

They don't decode gzip archives automatically.

HTTPThreadedDownloader and HTTPSThreadedDownloader - requests module.

url: http://docs.python-requests.org/en/latest/community/faq/

quote:
Requests automatically decompresses gzip-encoded responses, and does its best to decode response content to unicode when possible.

It's a feature of the module. Sometimes it's not desirable.

code:
def _get_decoder(mode):
if mode == 'gzip':
return zlib.decompressobj(16 + zlib.MAX_WBITS)

return DeflateDecoder()

class HTTPResponse(io.IOBase):
...
def init(self, body='', headers=None, status=0, version=0, reason=None,
strict=0, preload_content=True, decode_content=True,
original_response=None, pool=None, connection=None):
...
def read(self, amt=None, decode_content=None, cache_content=False):
...
content_encoding = self.headers.get('content-encoding', '').lower()
if self._decoder is None:
if content_encoding in self.CONTENT_DECODERS:
self._decoder = _get_decoder(content_encoding)
...

It's not equal downloaders.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.