UnicodeDecodeError about python-gnupg HOT 7 CLOSED

Note that a workaround of the monkey-patch is to define gpgon and gpgoff functions and use them around any gpg calls. Presumably an approach like this could be used in a decorator that could be injected into python-gnupg.

If code like this is used when a package is loaded, it will wrangle this situation:

import codecs
default_strict_func = codecs.lookup_error('strict')
import gnupg
gpg = gnupg.GPG('/path/to/gpg')
gpg_strict_func = codecs.lookup_error('strict')
def gpgon(): codecs.register_error('strict', gpg_strict_func)
def gpgoff(): codecs.register_error('strict', default_strict_func)

from python-gnupg.

Thanks for posting this workaround!! I've expanded it so that you may use GPGSafe class instead of gnupg.GPG without having to manage anything.

gpg = GPGSafe(use_agent=False, homedir="~/.gnupg/") # (instead of gpg = gnupg.GPG(...))
gpg.sign(text)
...

https://gist.github.com/e3rd/45aed2e93ac20843b6790b6b642da396

(Since this issue remains closed, I've also created a pull request so that it is noted by project visitors.)

from python-gnupg.

Hello @Nando-bog! Thanks for reporting this bug.

I don't have a Mac to test it on, but on a Linux machine I get the following:

>>> from gnupg import GPG
>>> g = GPG(homedir='foobar')
>>> c = g.encrypt('hallo', '50CC7744')
>>> print c.data
-----BEGIN PGP MESSAGE-----

hIwDz4uqK8zd5zkBA/962lezKEAsh157nZsiR+KYd/PW1jdxPG2u1RD4BaSEpkGF
cUlIkJmpliC0qiYvjA2ssnP4DPQ582z4rYAWVmbGjbrBIuQ3FBJBWxWbCkbDqCyu
tFzoCFkmILRQo6DLNgjNtXZPHiqYrP9ll5BaeteE1ooroJ0x3YSDMxbayX61OtJA
Aa2ST3t7iBU6xe6vRr8+4n3stbAwYB2H0RDh5/S/buVJQCI0tbmVSwLxdLZwadFF
XEq8W1X7iWPcGEmKlOkEag==
=9Rge
-----END PGP MESSAGE-----
>>> c.status
'encryption ok'
>>> d = g.decrypt(c.data)
>>> d.data
'hallo'

This is with Python 2.7.6 and python-gnupg-1.2.6.

It could have something to do with locale settings in your terminal. python-gnupg tries to be smart and respect them when necessary, otherwise it defaults to one with utf8. Or, it could be because you did str(c). I'm not sure.

@Nando-bog, do you think you could try again, using c.data and d.data, rather than str(c) and str(d), please?

from python-gnupg.

I believe this issue was fixed in my fix/49-unicode-decode-on-readline branch, which I've merged into develop and will be available in the next upcoming version (1.3.3).

Please reopen if the issue persists.

from python-gnupg.

I got a lot of UnicodeDecode errors while doing gpg.recv_keys() for a bunch of keys. It seems they were imported in the pubring successfully though.

Here is a key that gave me one of these errors: A0D180F35F45D0A0FBED9CD36E68F80607AF1977

I was using the current pypi version, should I switch to the develop branch or is it unstable?

Exception in thread Thread-683:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 808, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 761, in run
    self.__target(*self.__args, **self.__kwargs)
  File ""/home/ba/env_deed/local/lib/python2.7/site-packages/gnupg/_meta.py", line 564, in _read_response
    line = stream.readline()
  File "/home/ba/env_deed/lib/python2.7/codecs.py", line 530, in readline
    data = self.read(readsize, firstline=True)
  File ""/home/ba/env_deed/lib/python2.7/codecs.py", line 477, in read
    newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf6 in position 0: invalid start byte

from python-gnupg.

This solution needs to be rethought. Changing this global callback is affecting other code, most notably the DataStax Cassandra driver. The exception in the following code is never raised, so what is already valid UTF-8 has its characters replaced with garbage: https://github.com/datastax/python-driver/blob/master/cassandra/cqltypes.py#L675

from python-gnupg.

This solution definitely needs to be rethought. It breaks MIME encodings of attachments even if I am not applying PGP to them. For example, the base64 representation of an image will be munged here:

import gnupg
gpg = gnupg.GPG('/path/to/gpg')

import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.image import MIMEImage

def send_my_email():
    msg = MIMEMultipart()
    msg['Subject'] = 'subject'
    msg['From'] = '[email protected]'
    msg['To'] = '[email protected]'
    with open('/tmp/image.jpg', mode='rb') as image_file:
        image = MIMEImage(image_file.read())
    msg.attach(image)
    s = smtplib.SMTP('smtp.gmail.com', 587)
    s.starttls()
    s.login('[email protected]', 'password')
    s.send_message(msg)
    s.quit()

from python-gnupg.