We are having a problem with authenticated fdsnws downloads. We run Seiscomp4.5.0 from the master branch and Python3.8 on Ubuntu 18.04.5 LTS.
sysop@eida:~$ seiscomp-python
Python 3.8.0 (default, Feb 25 2021, 22:10:10)
[GCC 8.4.0] on linux
HTTPAuthSessionWrapper.getChildWithDefault encountered unexpected error
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/twisted/web/_auth/wrapper.py",
line 162, in _login
d = self._portal.login(credentials, None, IResource)
File "/usr/lib/python3/dist-packages/twisted/cred/portal.py", line
119, in login
return maybeDeferred(self.checkers[i].requestAvatarId, credentials
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
320, in addCallback
return self.addCallbacks(callback, callbackArgs=args,
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
310, in addCallbacks
self._runCallbacks()
--- <exception caught here> ---
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python/seiscomp/fdsnws/dataselect.py", line 373,
in requestAvatar
self.__userdb.getAttributes(avatarId)),
File "/usr/local/bin/fdsnwsauth", line 164, in getAttributes
return self.__users[name][1]
builtins.KeyError: "b'ttz_Ye1Jyw-aJ-JxmXG2tn04'"
HTTPAuthSessionWrapper.getChildWithDefault encountered unexpected error
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/twisted/web/_auth/wrapper.py",
line 162, in _login
d = self._portal.login(credentials, None, IResource)
File "/usr/lib/python3/dist-packages/twisted/cred/portal.py", line
119, in login
return maybeDeferred(self.checkers[i].requestAvatarId, credentials
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
320, in addCallback
return self.addCallbacks(callback, callbackArgs=args,
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
310, in addCallbacks
self._runCallbacks()
--- <exception caught here> ---
File "/usr/lib/python3/dist-packages/twisted/internet/defer.py", line
653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python/seiscomp/fdsnws/dataselect.py", line 373,
in requestAvatar
self.__userdb.getAttributes(avatarId)),
File "/usr/local/bin/fdsnwsauth", line 164, in getAttributes
return self.__users[name][1]
builtins.KeyError: "b'z7vvsZcSkQkRE0No2O3CEe6K'"
I figured out the problem is the character encoding of "name" passed to getAttributes() in fdsnws.py on line 165.
The name is expected to be a byte array of the user name. But instead it is a string that contains the b at the beginning and the two single quotes.
We have a workaround that removes the b and the two single quotes and makes it work as expected.
def getAttributes(self, name):
name = name[2:len(name)-1].encode('ascii') # ugly workaround to remove b'...' around name and convert to a correct ascii byte array
return self.__users[name][1]
But I am sure that is the wrong place for the fix. I can't figure out where the name gets messed up.
I have also tried to modify dataselect.py line 373 to use utils.py3ustr(avatarId) instead of only avatarId, but that didn't work either, giving the error "'str' object has no attribute 'decode'". I then gave up because I couldn't figure out who calls requestAvatar().
I am not sure if the error is in Seiscomp, or in one of the used python libraries. I tried both Twisted from the Ubuntu packages and also from pip3 install with the same issue.