mozilla / pyfxa Goto Github PK

The calls to cache.set happen only in the "happy path" when we've successfully verified a token. If we fail to verify a token for any reason, we usually throw an exception, which passes outside this control flow path. Some types of exception are probably indicative of a transient failure that we should retry, like a network failure, but other types represent rejected tokens (like this OutOfProtocolError), and those responses should maybe be cached too to minimize network traffic in the future.

Trying to log in to accounts.firefox.com based on the example given in the README.

>>> fxa.__version__
'0.7.4'
>>> from fxa.core import Client as FxAClient
>>> sess=FxAClient("https://api.accounts.firefox.com").login(email, passw)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/csaba/python/src/PyFxA/fxa/core.py", line 93, in login
    resp = self.apiclient.post(url, body)
  File "/home/csaba/python/src/PyFxA/fxa/_utils.py", line 335, in post
    raise fxa.errors.OutOfProtocolError(msg.format(resp.status_code))
  File "/home/csaba/python/src/PyFxA/fxa/_utils.py", line 321, in request
    msg = "API responded with non-integer timestamp: {0}"
fxa.errors.ClientError: The request was blocked for security reasons

However, if we patch the login method to consume further options like this:

diff --git a/fxa/core.py b/fxa/core.py
index 6706cdf..7b2a164 100644
--- a/fxa/core.py
+++ b/fxa/core.py
@@ -74,7 +74,7 @@ class Client(object):
             auth_timestamp=resp["authAt"],
         )
 
-    def login(self, email, password=None, stretchpwd=None, keys=False, unblock_code=None):
+    def login(self, email, password=None, stretchpwd=None, keys=False, unblock_code=None, **kw):
         stretchpwd = self._get_stretched_password(email, password, stretchpwd)
         body = {
             "email": email,
@@ -87,6 +87,9 @@ class Client(object):
         if unblock_code:
             body["unblockCode"] = unblock_code
 
+        for k,v in kw.items():
+            body[k] = v
+
         resp = self.apiclient.post(url, body)
         # XXX TODO: somehow sanity-check the schema on this endpoint
         return Session(

then login can be got to work with a suitable invocation:

>>> sess=FxAClient("https://api.accounts.firefox.com").login(email, passw, reason="login", verificationMethod="email-captcha")
>>> sess
<fxa.core.Session object at 0x7f5079b96280>

This is a recent change on the server side; in 2020 May email and passw was sufficient to log in.

I have tried to update to version 0.0.5 of PyFxA, but after having done so, I get an import error when I try to import Client.

Here's a stack trace:

>   from fxa.core import Client
../../.virtualenvs/marketplace-tests-unpin-fxapom/lib/python2.7/site-packages/fxa/core.py:9: in <module>
>   import browserid.jwt
E   ImportError: No module named jwt

I have created a new virtualenv and installed the package from PyPI, along with a number of others. Here is the result of pip freeze in my virtualenv:

Marketplace==0.9.1
PyBrowserID==0.9.2
PyFxA==0.0.5
PyYAML==3.10
UnittestZero==0.2.0
WebOb==1.4
argparse==1.3.0
certifi==0.0.8
cffi==0.9.2
chardet==2.1.1
cryptography==0.8.1
enum34==1.0.4
execnet==1.1
fxapom==1.2
hawkauthlib==0.1.1
httplib2==0.9
linecache2==1.0.0
mock==1.0.1
oauth2==1.5.211
py==1.4.15
pyasn1==0.1.7
pycparser==2.10
pytest==2.3.5
pytest-mozwebqa==1.6
pytest-xdist==1.8
requests==2.4.3
responses==0.3.0
selenium==2.45.0
six==1.9.0
traceback2==1.4.0
unittest2==1.0.1
wsgiref==0.1.2

I do have PyBrowserID installed, as can be seen from the above, so I'm not really sure what is causing the issue. To try to reproduce the issue I suggest creating a new virtualenv and installing PyFxA from PyPI to see if the same issue occurs.

It sometimes helps operational teams narrow down what application is the one causing load. Some clients will provide a client ID, but I think for token verification, it isn't required, so some other mechanism would be good, and User-Agent header seems like a good, obvious one that is visible at a high level.

To guard against DOS attacks we should put a hard upper bound on the size of the oauth verification cache.

This pull request hardcodes the jwks url we use for expediency:

#80

We should instead fetch the openid-configuration url to get the jwks url.

I am trying to update and run tests on openSUSE and following things are bit pita:

unittest2 are slowly phased out also they are only imported on py2.6 based on fxa/tests/utils.py thus there is no reason to import/require it every time (those requires should be limited every time, ie to pull in the unittest2 it should be done like unittest2;python_version<"2.7"'

Instead of nose executor it is good idea to use working python setup.py test or switch to pytest (simple swap from nose should be mostly enough).

Grequests should be replaced by py3 compatible requests-threads or requests-futures in test_monkey_patch_for_gevent.

For the online tests they should be skipped ie if you switch to pytest you can use @pytest.mark.online and we can later use pytest -m 'not online' for following tests:

[    3s] fxa/tests/test_core.py::TestCoreClient::test_account_creation_with_key_fetch FAILED [  5%]
[    3s] fxa/tests/test_core.py::TestCoreClient::test_account_login FAILED        [  6%]
[    4s] fxa/tests/test_core.py::TestCoreClient::test_email_code_verification FAILED [  7%]
[    4s] fxa/tests/test_core.py::TestCoreClient::test_forgot_password_flow FAILED [  8%]
[    4s] fxa/tests/test_core.py::TestCoreClient::test_get_random_bytes FAILED     [  9%]
[    4s] fxa/tests/test_core.py::TestCoreClient::test_resend_verify_code FAILED   [ 10%]
[    4s] fxa/tests/test_core.py::TestCoreClient::test_send_unblock_code FAILED    [ 11%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_change_password FAILED [ 12%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_email_status FAILED  [ 13%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_get_identity_assertion FAILED [ 14%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_get_identity_assertion_accepts_service FAILED [ 15%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_get_identity_assertion_handles_duration FAILED [ 16%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_get_random_bytes FAILED [ 16%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_session_status FAILED [ 17%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_sign_certificate FAILED [ 18%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_sign_certificate_handles_duration FAILED [ 19%]
[    4s] fxa/tests/test_core.py::TestCoreClientSession::test_totp FAILED          [ 20%]

You should not import mock on python3 as there it is regular part of the unittest:

[    4s] fxa/tests/test_requests_auth_plugin.py:5: in <module>
[    4s]     from fxa.tests.mock_utilities import (
[    4s] fxa/tests/mock_utilities.py:1: in <module>
[    4s]     import mock
[    4s] E   ModuleNotFoundError: No module named 'mock'

In some parts of the tests you already check for it but here it is hardcoded.

sync-1-us-west1-g.sync.services.mozilla.com

To support the larger efforts to get the FxA Rust Component into Desktop, we first need to get Sync completely migrated away from BrowserID over to OAuth.

Since requests-hawk update to 1.1.0, I see this in my logs:

[ERROR] 2021-01-02 00:03:16 SyncWorker::__pull(): payload content and/or content_type cannot be empty when always_hash_content is True

In case of calling /authorization using code-challenge, the OAuth client does not check the state parameter in the redirect URL provided by the server.

Following this article: http://www.servicedenuages.fr/en/load-testing-a-http-api-which-uses-oauth-for-authentication

I have been starting to work on tools to help load-testing with Firefox Account.

So far I have two command that let you write down a bearer token in a bash file (in order to export it in the environment to then load it)

The gist of the idea is there: https://gist.github.com/Natim/f4b1be4dde6eff8adaf2

What do you think about adding those helpers to PyFxA?
Should I put the function only (create_new_user, get_bearer_token)? Add the commands as well (bin/create_new_user_token loadtest_salt, bin/get_bearer_token email password)?
Do you want a specific file architecture for that? (fxa.commands.get_bearer_token)

In that case I would refactor the plugin to use functions that would be shared with the CLI tools.

The following script, when run with appropriate SYNC_EMAIL and SYNC_PASSWORD environment variables, is expected to fail with fxa.errors.ClientError: Unverified account.

import os
from fxa.core import Client

client = Client("https://api.accounts.firefox.com")

session = client.login(os.environ['SYNC_EMAIL'], os.environ['SYNC_PASSWORD'], keys=True)

email_status = session.get_email_status()
print email_status

session.fetch_keys()

And for me, it does indeed fail every time. However it's been reported that, after completing sign-in confirmation once, the script stops failing on subsequent invocations. All subsequent attempts to fetch keys succeed and email_status indicates that sessionVerified is true.

To be clear, concrete steps to reproduce are:

1. Set SYNC_EMAIL and SYNC_PASSWORD environment variables for your account.
2. Run the script, expect it to fail, see it fail.
3. Check your email, then follow the sign-in confirmation link after it arrives.
4. Run the script again, expect it to fail, see it succeed instead.

This was reported in IRC by @ecksun, who also mentioned that they don't see equivalent behaviour when doing the same thing using fxa-js-client. Is there something peculiar to PyFxA going on here?

Allow for the new PKCE protocol to work with OAuthClient as soon as no client_secret is provided.

I'm having some login issues and getting an error message each time I log in:

fxa.errors.ClientError: The request was blocked for security reasons

The javascript client seems to have a way to log in via unblock codes, as per:
https://github.com/mozilla/fxa-js-client/blob/ac623f807d64a566d4895cf6f6914eaa5a96d998/client/FxAccountClient.js#L238
https://github.com/mozilla/fxa-js-client/blob/ac623f807d64a566d4895cf6f6914eaa5a96d998/client/FxAccountClient.js#L1246

It would be great if this were also supported via the python client.

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

Python 2.6 is no longer supported by the Python core team, please upgrade your Python.

This is the DeprecationWarning coming while running tests on travis with Python 2.6

When i using kinto-fxa. I encountered some issues. After some digging , i found that it may caused by PyFxa.

"at+JWT" in
https://github.com/mozilla/fxa/blob/e440dae101bbd5d90e26dfcf60bea3edde8f362a/packages/fxa-auth-server/lib/oauth/jwt_access_token.js#L13
do not match in case

should be

if jwt.get_unverified_header(token).get('typ').lower() != 'at+jwt': 

I am confused by the oauth Client method names.

in trade_code, what is "code". Is it the session token issued during the connection or something else ?

there are several spots where "code" and "token" are used to describe the same thign I think. Can we clarify the term and use a single one everywhere ?

Thx

Add a way to define the requests driver to use in the apiclient.

https://github.com/mozilla/PyFxA/blob/master/fxa/_utils.py#L27-L28

When running the self tests using the pypi source distfile, many fail because of three missing files:

fxa/tests/bad-key.json
fxa/tests/jwks.json
fxa/tests/private-key.json

Please include them in the distfile.

Also, many more fail because the test server returns 502, for example:

_________________________________________________________________________________ TestCoreClient.test_account_creation _________________________________________________________________________________
                                                                                                                                                                                                        
self = <fxa.tests.test_core.TestCoreClient testMethod=test_account_creation>                                                                                                                            
                                                                                                                                                                                                        
    def test_account_creation(self):                                                                                                                                                                    
        acct = TestEmailAccount()                                                                                                                                                                       
        acct.password = DUMMY_PASSWORD                                                                                                                                                                  
>       session = self.client.create_account(acct.email, DUMMY_PASSWORD)                                                                                                                                
                                                                                                                                                                                                        
fxa/tests/test_core.py:58:                                                                                                                                                                              
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
fxa/core.py:64: in create_account                                                                                                                                                                       
    resp = self.apiclient.post(url, body)                                                                                                                                                               
fxa/_utils.py:338: in post                                                                                                                                                                              
    return self.request("POST", url, json, **kwds)                                                                                                                                                      
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
                                                                                                                                                                                                        
self = <fxa._utils.APIClient object at 0x761ebf01cc20>, method = 'POST', url = 'https://stable.dev.lcip.org/auth/v1/account/create'                                                                     
json = {'authPW': '567643e5ada89e9fc87f1f3b4e8d75a7c6faaa7d7695967c924588a8224b2396', 'email': '[email protected]'}, retry_auth_errors = True                                                
kwds = {'headers': {'User-Agent': 'Mozilla/5.0 (Mobile; Firefox Accounts; rv:1.0) PyFxA/0.7.7 python-requests/2.31.0'}, 'timeout': 30}, resp = <Response [502]>                                         
headers = {'User-Agent': 'Mozilla/5.0 (Mobile; Firefox Accounts; rv:1.0) PyFxA/0.7.7 python-requests/2.31.0'}, content_type = 'text/html', msg = 'API responded with non-json content-type: {0}'        
...
        # Everything should return a valid JSON response.  Even errors.                                                                                                                                 
        content_type = resp.headers.get("content-type", "")                                                                                                                                             
        if not content_type.startswith("application/json"):                                                                                                                                             
            msg = "API responded with non-json content-type: {0}"                                                                                                                                       
>           raise fxa.errors.OutOfProtocolError(msg.format(content_type))                                                                                                                               
E           fxa.errors.OutOfProtocolError: API responded with non-json content-type: text/html                                                                                                          
                                                                                                                                                                                                        
fxa/_utils.py:270: OutOfProtocolError                                                                                                                                                                   

No idea how to fix that part.

This should be a method on the core.Client class, like make_assertion or similar. It can call PyBrowserID for all the hard work, we just need to do a bit of certificate management on the class.

We should make this lib python 3 compatible - that'd be cool :)

I am suddenly seeing issues where, when I create an account, I am not receiving the verification email. I am calling wait_for_email in https://github.com/mozilla/PyFxA/blob/master/fxa/tests/utils.py#L99.

This is causing all of our tests which require an account to fail, so is a pretty high priority.

cc: @stephendonner

STR:

1. Install pyamo from https://github.com/kewisch/pyamo
2. Make sure you are logged in to Firefox Sync in Firefox
3. Run amo info lightning (*)

Result:

• Firefox shows a popup notification "Firefox is now syncing with (null)"

Expected:

• No notification, or something like "Another device has logged in to Firefox Sync"

Other Details:
Happens both with PyFxA 0.3.0 from pip and latest master.

The code that calls PyFxA can be found here: https://github.com/kewisch/pyamo/blob/master/pyamo/utils.py#L62

(*) This command will likely fail with a message that you don't have permissions, but the notification in Firefox appears just after entering your password for me. If you want to test a command that works with a normal AMO account then you can use amo upload, but that will actually upload an xpi file to one of your add-ons.

We're using json argument in a post request, which was added in 2.4.2 so the package dependency should be >= 2.4.2. See http://docs.python-requests.org/en/latest/community/updates/#id8

@bobsilverberg I believe this explains the failures on our CI.

Any client with the ability to generate assertions on behalf of the user should also have the ability to generate oauth tokens. The oauth-server exposes the necessary APIs for this so we just need to plug them into the client API.

The tests currently talk to a live server, which is great for dev testing but not ideal for an automated suite. We should change them to use mocked responses by default, but still be able to talk to a live server if requested explicitly.

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

Following @rfk comments here: https://github.com/mozilla-services/loop-server/pull/359/files#r44236297

We should probably add a way to expose API error codes at the library level.

I haven't yet been able to track down the exact circumstances but we ran into this issue in basket. Sometimes the URL returned by the get_redirect_url() method would look like https://oauth.stage.mozaws.net/v1//authorization?scope=profile... and this would result in a 404 response. Seems to happen regardless of the server_url value. I was using the values for server_url straight from the constants.py file for the oauth key. I'll update this if I figure anything else out. My solution for now is to build the URL myself in the basket code.

When using PyFxa, I can't set User agent even trying to overwrite value by:
fxa_client.apiclient.headers["User-Agent"] = "Eolie-Web-Browser"

In Firefox account console, I see only Python Requests 2.13.

In _utils.py, I see Fxa should set a custom user agent.

I am getting this warning when I run a pytest script that uses fxa.tests.utils.TestEmail account

cannot collect test class 'TestEmailAccount' because it has a init constructor

Running in this environment: platform darwin -- Python 3.6.3, pytest-3.3.2, py-1.5.2, pluggy-0.6.0

When I try examples from README, I get:

raw_resp = requests.get('https://profile.accounts.firefox.com/v1/profile',
... auth=FxABearerTokenAuth(email, password,
... ['profile'], "browser"))
Traceback (most recent call last):
File "", line 3, in
File "/usr/lib/python3.6/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 474, in request
prep = self.prepare_request(req)
File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 407, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "/usr/lib/python3.6/site-packages/requests/models.py", line 306, in prepare
self.prepare_auth(auth, url)
File "/usr/lib/python3.6/site-packages/requests/models.py", line 543, in prepare_auth
r = auth(self)
File "/usr/lib/python3.6/site-packages/fxa/plugins/requests.py", line 146, in call
oauth_server_url=self.oauth_server_url)
File "/usr/lib/python3.6/site-packages/fxa/tools/bearer.py", line 42, in get_bearer_token
client_id)
File "/usr/lib/python3.6/site-packages/fxa/oauth.py", line 154, in authorize_token
resp = self.apiclient.post(url, body)
File "/usr/lib/python3.6/site-packages/fxa/_utils.py", line 300, in post
return self.request("POST", url, json, **kwds)
File "/usr/lib/python3.6/site-packages/fxa/_utils.py", line 286, in request
raise fxa.errors.ClientError(body)
fxa.errors.ClientError: Invalid request parameter

Currently, a failing cache prevents the OAuth verification to complete. Shouldn't it just fail to cache the response instead?

In other words, shouldn't we catch any exception during cache.set() ?

See suggestion in mozilla-services/readinglist#121 (comment)