Comments (6)
sserrata
commented on May 23, 2024
Concerning the HTTPClient auto_refresh attribute/kwarg, the docstring will be updated/corrected to reflect the true behavior:
"Perform token refresh prior to request if access_token is None or expired. Defaults to True"
from pan-cortex-data-lake-python.
sserrata
commented on May 23, 2024
Furthermore, in the request method, on line 223, there's credentials = kwargs.pop("credentials", None), where it should probably be credentials = kwargs.pop("credentials", self._credentials).
With respect to kwargs.pop("credentials", None), this was changed in order to avoid inadvertently applying credentials twice. In controlled tests, I've observed auto_refresh behaving as expected. Please see the test steps below:
- Generate credentials profile with
client_id,client_secretandrefresh_token - Attempt an API request and observe initial
auto_refreshbehavior due toNonecondition (see below)
Note: an extra INFO log was added to log the headers value in the request method
(env) DFWMAC1P6LVDN:examples sserrata$ ./test.py
DEBUG:pan_cortex_data_lake.httpclient:Default headers applied: {'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:pan_cortex_data_lake.httpclient:Default headers applied: {'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:pan_cortex_data_lake.httpclient:Applying session-level credentials
INFO:pan_cortex_data_lake.httpclient:{'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.paloaltonetworks.com:443
DEBUG:urllib3.connectionpool:https://api.paloaltonetworks.com:443 "POST /api/oauth2/RequestToken HTTP/1.1" 200 630
DEBUG:pan_cortex_data_lake.httpclient:Token refreshed due to 'None' condition
DEBUG:pan_cortex_data_lake.httpclient:Credentials applied to authorization header
INFO:pan_cortex_data_lake.httpclient:{'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'Bearer eyJ...'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.us.cdl.paloaltonetworks.com:443
DEBUG:urllib3.connectionpool:https://api.us.cdl.paloaltonetworks.com:443 "POST /query/v2/jobs HTTP/1.1" 201 108
- Manually edit the credentials.json profile to replace existing
access_tokenwith an expired token. - Attempt API request again and observe
auto_refreshbehavior, this time refreshing due to "Expired" condition.
DEBUG:pan_cortex_data_lake.httpclient:Default headers applied: {'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:pan_cortex_data_lake.httpclient:Default headers applied: {'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:pan_cortex_data_lake.httpclient:Applying session-level credentials
INFO:pan_cortex_data_lake.httpclient:{'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.paloaltonetworks.com:443
DEBUG:urllib3.connectionpool:https://api.paloaltonetworks.com:443 "POST /api/oauth2/RequestToken HTTP/1.1" 200 630
DEBUG:pan_cortex_data_lake.httpclient:Token refreshed due to 'expired' condition
DEBUG:pan_cortex_data_lake.httpclient:Credentials applied to authorization header
INFO:pan_cortex_data_lake.httpclient:{'User-Agent': 'cortex-data-lake-python/2.0.0-a9', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'Bearer eyJ...'}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.us.cdl.paloaltonetworks.com:443
DEBUG:urllib3.connectionpool:https://api.us.cdl.paloaltonetworks.com:443 "POST /query/v2/jobs HTTP/1.1" 201 108
As best as I can tell, everything appears to be working as expected. Please let me know if I'm missing something. Thanks!
from pan-cortex-data-lake-python.
mishas
commented on May 23, 2024
Hello @sserrata,
In the test proposed above, you're recreating the object (and in that case, it indeed refreshes the token). But if you keep holding the object - it doesn't:
c = Credentials(refresh_token=MY_KEY, client_id=MY_CLIENT_ID, client_secret=MY_CLIENT_SECRET)
qs = QueryService(url="https://api.us.cdl.paloaltonetworks.com", credentials=c)
q = qs.create_query(query_params=query_params) # This will work
time.sleep(60*60+ 5*60) # Sleep for 1 hour and 5 minutes
q = qs.create_query(...) # This will not work.
from pan-cortex-data-lake-python.
sserrata
commented on May 23, 2024
Hi @mishas - yes, I understand that after an hour the access_token would have expired. I'm attempting to simulate this by manually replacing the credentials.json access_token with an expired value. I will try to replicate using the same method you outlined above and will report back with the results (in ~1 hour).
from pan-cortex-data-lake-python.
mishas
commented on May 23, 2024
You can simulate this by calling:
qs._httpclient.session.headers.update({"Authorization": "Bearer {}".format(old_token)})
instead of sleep.
(i'm not 100% sure about the above, didn't try it, but I think it should work)
from pan-cortex-data-lake-python.
sserrata
commented on May 23, 2024
Hi @mishas - please see PR #152 for a fix. I discovered a regression bug introduced in #140. In weighing different options, I realized that it made little sense to apply credentials at the session/HTTPClient layer, if they would ultimately get applied at the request layer. Thanks again for reporting this issue.
from pan-cortex-data-lake-python.
Related Issues (20)
- Add MemoryStorage support to TinyDBStore adapter HOT 1
- Add support for Event Service API flush endpoint
- _apply_credentials doesn't respect auto_refresh parameter HOT 1
- Update docs HOT 1
- Update examples HOT 1
- Support for Developer Tokens
- Support for regions HOT 1
- Invalid comparison leads to "PENDING" state returning False
- Refresh incorrectly attempted when both static access_token and developer_token are present HOT 3
- Cleanup unnecessary use of json() property
- Incorrect clientType and clientVersion defined in create_query() method
- Subsequent refresh() attempts fail when using developer_token
- iter_job_results() overrides credentials in get_job_results()
- Raised exception in iter_job_results() prevents API errors from returning
- Proposed changes to default storage adapter HOT 2
- iter_job_results() yields for waiting states
- More flexible python requirements? HOT 5
- [Community Health Assessment] Changes needed HOT 2
- Developer token auth doesnt work anymore HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pan-cortex-data-lake-python.