akamai / akamaiopen-edgegrid-python Goto Github PK

solution will be similar to this one: https://github.com/kennethreitz/requests/issues/1979.

test with /papi/v0/properties/{prpId}/versions/latest

Hi Master,

When you release new version, please include wheel, otherwise Windows user have a issue as follows.

building '_cffi_backend' extension
error: Unable to find vcvarsall.bat

Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cryptography cffi
Installing collected packages: cffi, cryptography

Running setup.py install for cffi ... error

Sincerely,
Hwaseob

I am unable to parse the contents of the default section successfully.
The .edgerc file is readable by the CLI and HTTPie
This seems to be an issue similar to https://github.com/akamai/AkamaiOPEN-edgegrid-python/issues/48

Error message:
% /Applications/Xcode.app/Contents/Developer/usr/bin/python3 "/Users/redacted/Apps/edge.py"
Traceback (most recent call last):
File "/Users/redacted/Apps/edge.py", line 6, in
baseurl = 'https://%s' % edgerc.get(section, 'host')
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/configparser.py", line 781, in get
d = self._unify_values(section, vars)
File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/configparser.py", line 1149, in _unify_values
raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'default'

The edge.py file(renamed to .txt) is as attached.

edge.txt

I have installed the httpie-edgegrid from the github link provided in Akamai documentation. After running the setup.py file in a Linux (ubuntu) environment I have set up in my PC, the .edgerc file, which supposed to be generated in my home Desktop is no where to be found.

Therefore I cannot setup my credentials.
Any solution or alternative would be much appreciated.

'/diagnostic-tools/v1/locations' is a deprecated endpoint that response with 410 "Gone" this should be updated to a different example endpoint, ideally something that requires auth but all API tokens have access to

The latest version of HTTPie (v2.3.0) fails when making POST request using the Edgegrid authentication plugin which uses https://github.com/akamai/AkamaiOPEN-edgegrid-python to make the underlaying calls.

I enabled debugging and tracked the failure to the Edgegrid python library (edgegrid.py line 146)

Here is a command I used to reproduce the issue:
$ http --verbose -A edgegrid --debug -a devrel-all: POST :/config-dns/v2/zones/akamaideveloper.net/zone-file Content-Type:text/dns < akamaideveloper.net.zone

Here are the last few lines of the error output that point to the issue:

  File "/Users/jgarza/.pyenv/versions/3.8.0/lib/python3.8/site-packages/akamai/edgegrid/edgegrid.py", line 147, in make_content_hash
    if r.method == 'POST' and len(prepared_body) > 0:
TypeError: object of type '_io.BufferedReader' has no len()

I suspect the new version of HTTPie streams the uploads, and therefore the prepared_body has no length when evaluated (it is possible this may affect other CLI tools that leverage this library that try to stream uploads).

I am running Ubuntu in which Python 2.7 and 3.4 coexist, but 2.7 is the default. edgegrid-python requires Python3 to install. pip3 is Python3's own installer. However, even with pip3, edgegrid-python setup still complains about Python not being version 3:

$ /usr/bin/python -V
Python 2.7.6
$ pip3 -V
pip 1.5.4 from /usr/lib/python3/dist-packages (python 3.4)
$ pip3 install edgegrid-python

Downloading/unpacking edgegrid-python
Downloading edgegrid-python-1.1.0.tar.gz
Running setup.py (path:/tmp/pip_build_mcgitty/edgegrid-python/setup.py) egg_info for package edgegrid-python

Downloading/unpacking requests>=2.3.0pyopenssl (from edgegrid-python)
Downloading requests-2.18.4-py2.py3-none-any.whl (88kB): 88kB downloaded
Downloading/unpacking ndg-httpsclient (from edgegrid-python)
Downloading ndg_httpsclient-0.4.3-py3-none-any.whl
Downloading/unpacking pyasn1 (from edgegrid-python)
Downloading pyasn1-0.3.7-py2.py3-none-any.whl (63kB): 63kB downloaded
Requirement already satisfied (use --upgrade to upgrade): urllib3 in /usr/lib/python3/dist-packages (from edgegrid-python)
Downloading/unpacking idna>=2.5,<2.7 (from requests>=2.3.0pyopenssl->edgegrid-python)
Downloading idna-2.6-py2.py3-none-any.whl (56kB): 56kB downloaded
Downloading/unpacking certifi>=2017.4.17 (from requests>=2.3.0pyopenssl->edgegrid-python)
Downloading certifi-2017.7.27.1-py2.py3-none-any.whl (349kB): 349kB downloaded
Downloading/unpacking chardet>=3.0.2,<3.1.0 (from requests>=2.3.0pyopenssl->edgegrid-python)
Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB): 133kB downloaded
Downloading/unpacking PyOpenSSL (from ndg-httpsclient->edgegrid-python)
Downloading pyOpenSSL-17.3.0-py2.py3-none-any.whl (51kB): 51kB downloaded
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /usr/lib/python3/dist-packages (from PyOpenSSL->ndg-httpsclient->edgegrid-python)
Downloading/unpacking cryptography>=1.9 (from PyOpenSSL->ndg-httpsclient->edgegrid-python)
Downloading cryptography-2.1.2.tar.gz (441kB): 441kB downloaded
Running setup.py (path:/tmp/pip_build_mcgitty/cryptography/setup.py) egg_info for package cryptography
error in cryptography setup command: Invalid environment marker: python_version < '3'
Complete output from command python setup.py egg_info:
error in cryptography setup command: Invalid environment marker: python_version < '3'

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_mcgitty/cryptography
Storing debug log for failure in /home/mcgitty/.pip/pip.log

pip.log.gz

Goal:

I am trying to setup ci/cd for my application using gitlab.

Background Info:

I am using Akamai API in my app using AkamaiOPEN-edgegrid-python. I have setup the Akamai api tokens in the Gitlab Environment variables section.

Problem

I have played around with my application locally multiple times and I did not run into any issues related to authorization issues.
When I setup the .gitlab-ci.yml to trigger the gitlab runner, I am seeing 401 error..

'title': 'Not authorized', 'status': 401, 'detail': 'The signature does not match'

I double checked the credentials for typo/newline issues in the environment variables section and everything looks correct.

It looks like the credentials are being accessed successfully but for some reason I see not authorized error.

I wanted to clarify if this is related to any other issue that I am not aware of in the authorization world or could it be due to the this package?

I did spend considerable amount of time troubleshooting this issue but no luck yet. Any help/guidance to troubleshoot this will be appreciated.

Thanks!

The version of pyOpenSSL included in base Mac OSX is incompatible with this library. As a result, users of egcurl get a misleading error message when they they try to run that script.

The version included with OSX is 0.13.1, which was released back in 2013. I am not sure what minimum version we really need, but I propose to set the minimum version in requirements.txt to 17.0.0, which was released in August 2017. The current version is 19.0.0, and we could reasonably pick that one too if we wanted.

It's worth noting that I think we only need this library if the user is using Python 2.7. There are some if statements that only import it if the Python major version is less than 3. Since Python 3 is more than 10 years old now it'd really be nice if Apple would upgrade their base Python version...

In the mean time, Mac OS users can upgrade their local pyOpenSSL version with this command:

pip install -U pyOpenSSL

Note that this is NOT run with sudo - that's very important. The -U argument instructs pip to install it into the user's home directory. You CANNOT successfully replace the system version of pyOpenSSL. The command will fail.

I've just updated my homebrew python3.6 to 3.7. Now I'm getting an error for the default config values:
Traceback (most recent call last): File "./egcurl", line 333, in <module> sys.exit(main()) File "./egcurl", line 254, in main config = get_config(args.eg_edgerc, args.eg_config, section) File "./egcurl", line 85, in get_config config = _parse_edgerc(os.path.expanduser(edgerc_filename), section) File "./egcurl", line 91, in _parse_edgerc edgerc = EdgeRc(file) File "/usr/local/lib/python3.7/site-packages/akamai/edgegrid/edgerc.py", line 36, in __init__ ConfigParser.__init__(self, {'max_body': '131072', 'headers_to_sign': None}) File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 638, in __init__ self._read_defaults(defaults) File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1216, in _read_defaults self.read_dict({self.default_section: defaults}) File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 753, in read_dict self.set(section, key, value) File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1197, in set self._validate_value_types(option=option, value=value) File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1182, in _validate_value_types raise TypeError("option values must be strings") TypeError: option values must be strings

In file edgerc.py I adjusted from:
ConfigParser.__init__(self, {'max_body': '131072', 'headers_to_sign': None})
to:
ConfigParser.__init__(self, {'max_body': '131072', 'headers_to_sign': 'None'})

Strange is that the edgerc.py file found in this repo is different to what I have. Seems the pip download is at a different version. I found my edgerc matching the branch support_python3. But there even is another one call python3_support.

I guess this needs a little cleaning :) So I'm not making a pull request without knowing which version to base to.

The source code has:
./akamai/edgegrid/test/sample_edgerc

However the release tarball on pypi doesn't have that file. I'm trying to package this project up for Ubuntu and can't run all of the unit tests as a result.

======================================================================
ERROR: test_edgerc_default (akamai.edgegrid.test.test_edgegrid.EGSimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/test/test_edgegrid.py", line 123, in test_edgerc_default
    auth = EdgeGridAuth.from_edgerc(os.path.join(mydir, 'sample_edgerc'))
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/edgegrid.py", line 120, in from_edgerc
    client_token=rc.get(section, 'client_token'),
  File "/usr/lib/python3.8/configparser.py", line 781, in get
    d = self._unify_values(section, vars)
  File "/usr/lib/python3.8/configparser.py", line 1149, in _unify_values
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'default'

======================================================================
ERROR: test_edgerc_broken (akamai.edgegrid.test.test_edgegrid.EGSimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/test/test_edgegrid.py", line 131, in test_edgerc_broken
    auth = EdgeGridAuth.from_edgerc(os.path.join(mydir, 'sample_edgerc'), 'broken')
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/edgegrid.py", line 120, in from_edgerc
    client_token=rc.get(section, 'client_token'),
  File "/usr/lib/python3.8/configparser.py", line 781, in get
    d = self._unify_values(section, vars)
  File "/usr/lib/python3.8/configparser.py", line 1149, in _unify_values
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'broken'

======================================================================
ERROR: test_edgerc_headers (akamai.edgegrid.test.test_edgegrid.EGSimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/test/test_edgegrid.py", line 145, in test_edgerc_headers
    auth = EdgeGridAuth.from_edgerc(os.path.join(mydir, 'sample_edgerc'), 'headers')
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/edgegrid.py", line 120, in from_edgerc
    client_token=rc.get(section, 'client_token'),
  File "/usr/lib/python3.8/configparser.py", line 781, in get
    d = self._unify_values(section, vars)
  File "/usr/lib/python3.8/configparser.py", line 1149, in _unify_values
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'headers'

======================================================================
ERROR: test_edgerc_from_object (akamai.edgegrid.test.test_edgegrid.EGSimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/test/test_edgegrid.py", line 149, in test_edgerc_from_object
    auth = EdgeGridAuth.from_edgerc(EdgeRc(os.path.join(mydir, 'sample_edgerc')))
  File "/build/python-edgegrid-1.1.1/akamai/edgegrid/edgegrid.py", line 120, in from_edgerc
    client_token=rc.get(section, 'client_token'),
  File "/usr/lib/python3.8/configparser.py", line 781, in get
    d = self._unify_values(section, vars)
  File "/usr/lib/python3.8/configparser.py", line 1149, in _unify_values
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'default'

----------------------------------------------------------------------
Ran 21 tests in 0.008s

FAILED (errors=4)

it looks like edgegrid depends on requests https://github.com/akamai-open/AkamaiOPEN-edgegrid-python/blob/master/setup.py#L12 but it looks like you might have to bump the version number to at least 2.3 kennethreitz/requests@0caa243 because the method is_redirect used here https://github.com/akamai-open/AkamaiOPEN-edgegrid-python/blob/master/akamai/edgegrid/edgegrid.py#L205 is not in version 1.2.2

$ [sudo] pip install edgegrid-python --upgrade

Command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_build_root/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-7t0hME-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/cffi
Storing debug log for failure in /home/maswanso/.pip/pip.log

I tried with/without sudo and with/without --upgrade.
It failed on a Mac (python 2.7.10) and on Ubuntu 14.04 (python 2.7.6).

Please add an explicit indication whether this package supports both python2 and python3. Ideally, this should be reflected in the classifiers parameter when your setup.py is calling setuptools.setup() (see https://packaging.python.org/guides/distributing-packages-using-setuptools/#classifiers and https://pypi.org/classifiers/). For example:

setup(
    ...
    classifiers=[
        'Programming Language :: Python :: 2',
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.7'
    ]
)

Also, it wouldn't hurt to add specific, enforceable, supported python versions into python_requires, for example:

    # Require Python 2.7.10+ or Python 3 starting with 3.7 (per PEP-440)
    python_requires='>=2.7.10, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, '
                    '!=3.5.*, !=3.6.*, <4',

When trying to download more than one configId at a time from the /siem/v1/configs/ route with my Python script I always get a 401 error.

I.E: /siem/v1/configs/111?from=1611249319&limit=200000 works, but /siem/v1/configs/111;222;333?from=1611249319&limit=200000 doesn't

The same configIds with the same edgerc config are correctly downloaded by the sample CEF connector at https://github.com/akamai/siem-cef-connector.

I suspect it's a Python Edgegrid authentication problem, since I analyzed the URL requests with Wireshark and compared my generated requests to the sample CEF connector ones, and they look identical, the only difference being (of course) the signature.

While trying to execute commands using below, i am stilling see errors.

http --auth-type edgegrid -a default: :/diagnostic-tools/v2/ghost-locations/available

My edgerc file exists in ~/.edgerc
Tried below as well pip3 install --ignore-installed pyOpenSSL but still no luck, can you explain why is this failing?

http --auth-type edgegrid -a default: :/diagnostic-tools/v2/ghost-locations/available
usage:
http [--auth-type {basic,bearer,digest}] [METHOD] URL [REQUEST_ITEM ...]

error:
argument --auth-type/-A: invalid choice: 'edgegrid' (choose from 'basic', 'bearer', 'digest')

for more information:
run 'http --help' or visit https://httpie.io/docs/cli

Describe the bug
SIEM API (https://techdocs.akamai.com/siem-integration/reference/get-configid) has a recipe called Fetch security event in time-based mode | Python that uses this library. It does not work and returns an error.

To Reproduce
Steps to reproduce the behavior:

import requests
from akamai.edgegrid import EdgeGridAuth, EdgeRc
from urllib.parse import urljoin

edgerc = EdgeRc('~/.edgerc.txt')
section = 'default'
baseurl = 'https://%s' % edgerc.get(section, 'host')

session = requests.Session()
session.auth = EdgeGridAuth.from_edgerc(edgerc, section)

configId = "12345"

path = "https://hostname/siem/v1/configs/{}".format(configId)

headers = {
    "Accept": "application/json"}

querystring = {
    "from": "1488816442",
    "to": "1488816784",
    "limit": 1000
}

result = session.get(urljoin(baseurl, path),
                     headers=headers, params=querystring)
print(result.status_code)
print(result.json())

Expected behavior
For the documentation example to work.

Actual behavior
It outputs this error.

{
'type': 'https://problems.cloudsecurity.akamaiapis.net/siem/v1/missing-parameters',
'title': 'Missing mandatory parameter(s) or illegal parameter parameter(s) in the request',
'instance': 'https://{host}/siem/v1/configs/{configid}?from={from}',
'detail': 'Missing mandatory parameter(s) or illegal parameter parameter(s) in the request',
'method': 'GET'
...
}

Judging from the instance above, it appears that only the first query parameter is being recognized. If the & character is encoded to %26 in the signature generation then the instance variable changes to 'instance': 'https://{host}/siem/v1/configs/{configid}?from={from}&to={to}&limit={limit}', but gives a 401 error for the signature not matching. If the & character is encoded to %26 in the request and signature, then it returns a 400 error for missing parameters.

The releases of edgegrid-python to PyPI contain the sources for the test suite; however, test data is not installed. That means the tests can't actually work.

Example:

Create a virtualenv and install edgegrid-python:

$ virtualenv-3 ~/edgegrid-env
$ . ~/edgegrid-env/bin/activate
$ pip3 install edgegrid-python

Installation included akamai.edgegrid.test package:

$ find /home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/edgerc.py
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/test_edgegrid.py
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/__pycache__
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/__pycache__/__init__.cpython-36.pyc
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/__pycache__/test_edgegrid.cpython-36.pyc
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/__init__.py
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/edgegrid.py
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/__pycache__
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/__pycache__/__init__.cpython-36.pyc
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/__pycache__/edgegrid.cpython-36.pyc
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/__pycache__/edgerc.cpython-36.pyc
/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/__init__.py

But it did not include testdata.json, so the tests aren't actually usable:

$ python3 -munittest akamai.edgegrid.test.test_edgegrid
E
======================================================================
ERROR: akamai.edgegrid.test.test_edgegrid (unittest.loader._FailedTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/test_edgegrid.py", line 222, in load_tests
    return suite()
  File "/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/test_edgegrid.py", line 199, in suite
    with open("%s/testdata.json" % mydir) as testdata:
FileNotFoundError: [Errno 2] No such file or directory: '/home/rmcgover/edgegrid-env/lib/python3.6/site-packages/akamai/edgegrid/test/testdata.json'

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)

Steps to reproduce:

• pip install edgegrid-python

Actual behavior:

• Installs unusable tests

Expected behavior:

• Either installs the tests plus everything needed to run them;
• Or does not install tests at all

It appears that in Python 3.8 the module is not recognizing the sections ?

MacOS 10.15

cat ~/.edgerc
[default]
client_secret = the-client-secret
host = secretlocation.luna.akamaiapis.net
access_token = the-access-token
client_token = the-client-token

section = 'default'
baseurl = 'https://%s' % edgerc.get(section, 'host')
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/Cellar/[email protected]/3.8.2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/configparser.py", line 781, in get
d = self._unify_values(section, vars)
File "/usr/local/Cellar/[email protected]/3.8.2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/configparser.py", line 1149, in _unify_values
raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'default'

The max-body value is hard coded to 2048 in both edgerc.py and edgegrid.py. Please update the library to honor the max-body value set in .edgerc.

This is causing issues with large POST bodies like the ones seen when using the config-dns OPEN API.

I am able to use EdgeGridAuth and Session to make GET and POST requests for a domain dns domain with small amount of entries.

But as soon as i added more entries which makes the post body exceeded 2048, i starts to see following the in the logs and 'status_code': 401,"detail": "The signature does not match",

DEBUG:akamai.edgegrid.edgegrid:data length 2109 is larger than maximum 2048
DEBUG:akamai.edgegrid.edgegrid:data truncated to 2048 for computing the hash

This isnt an issue, rather a question. Can we use aiohttp instead of requests library to support asynchronous http calls ?