scanapi / scanapi Goto Github PK

Problem

If we define a var containing an upper case letter, like:

vars:
    apiKey: abc123

and we try to use it later as ${apiKey} it will raise the error:

ERROR:scanapi.evaluators.string_evaluator:'apiKey' environment variable not set or badly c

Possible Solution

To check if there is any lower case in the word here:

https://github.com/camilamaia/scanapi/blob/master/scanapi/evaluators/string_evaluator.py#L48

if any(letter.islower() for letter in variable_name):
  continue

Description

We need to ensure that API spec has some mandatories keys in order to work properly.

The first mandatory key that need to be checked is the key api. The specification should start with it.
https://github.com/scanapi/scanapi/blob/master/scanapi/__init__.py#L68

Under the key endpoints, we need to ensure each entry has at least a name and a requests key
https://github.com/scanapi/scanapi/blob/master/scanapi/tree/endpoint_node.py#L79

Under the key requests, we need to ensure each entry has at least a name and a path key.
https://github.com/scanapi/scanapi/blob/master/scanapi/tree/request_node.py#L106

This is an example of a minimal possible structure:

api:
  endpoints:
    - name: scanapi-demo
      requests:
        - name: health
          path: http://demo.scanapi.dev/api/health/

If any of this mandatories keys is missing, an error should be raised.

Similar to headers implementation

api:
  base_url: ${BASE_URL}
  headers:
    Authorization: ${BEARER_TOKEN}
  params:
    per_page: 10

Description

At PyPi page desccription the image scanapi-report-example.png is not loading

This happens because it is being used the relative path to link the image on README.md

How it is now:

<p align="center">
  <img src="images/scanapi-report-example.png" width="700">
</p>

How it is should be:

<p align="center">
  <img src="https://github.com/camilamaia/scanapi/blob/master/images/scanapi-report-example.png" width="700">
</p>

Enable to have API spec in multiples files using an include syntax.

Current the format is markdown.

💄

• Show only path_url in the titles
• Add response and request headers
• Add request headers and body
• Add response headers, metada and body
• Add cURL
• Add response time

Description

Make coverage > 90% for each file. You can find the current % for each file here: https://codecov.io/gh/scanapi/scanapi/tree/master/scanapi

Create a Makefile with the following commands:

• Install
• Test

Current, when there is an error when executing a command inside the python code tag ${{ }}, this is the message:

ERROR:scanapi.evaluators.string_evaluator:Invalid Python code defined in the API spec: Expecting value: line 1 column 1 (char 0)

We should improve it to show precisely which error happened.

Generate automated code documentation from the docstrings.

https://wiki.python.org/moin/DocumentationTools
https://www.sphinx-doc.org/en/master/

Currently we are evaluating python code from API specification ${{ code }} using eval:

We must use a safer and more elegant solution. I am out of ideas here.

• Templates (like jinja) don't work because we need to evaluate dynamically the python code while the requests are being made. The python code might depend on the last request response, for example.
• ast.literal_eval does not support access to external variables

Show Request Body, if it exists, on Markdown Report. Current it is showing only response body.

Description

Error when running scanapi v0.0.16

jinja2.exceptions.TemplateNotFound: markdown.jinja```

Error when running scanapi v0.0.15

    from scanapi.tree.api_tree import APITree
ModuleNotFoundError: No module named 'scanapi.tree'

Create a README header, something like:

https://github.com/barbosa/clorox

Some References: https://docs.google.com/document/d/1b8djucd09fjzzfGDsRYEyKa8c7VDZBgFVwJBr9Y52h8/edit?usp=sharing

factory_boy is a fixtures replacement based on thoughtbot’s factory_bot

As a fixtures replacement tool, it aims to replace static, hard to maintain fixtures with easy-to-use factories for complex object.

Instead of building an exhaustive test setup with every possible combination of corner cases, factory_boy allows you to use objects customized for the current test, while only declaring the test-specific fields:

Description

Configure a changelog linter. Maybe https://github.com/rcmachado/changelog

Related with: #88

Something similar to the right column in the post docs:

Starting with curl is good enough 👍

Create ScanAPI spec from an OpenAPI specification

Create a command that converts an OpenAPI specification file into a ScanAPI specification file.
OpenAPI Specification: https://swagger.io/specification/

For that, we need to create a new Click command called convert in the __main__.py file.

With the arguments:

• input file path (required)
• output file path (required)

And with the options:

• -f, --from, input_format. (default should be openapi, required)
• -t, --to, output_format. (default should be scanapi, required)

Example:

$ scanapi convert -f openapi -t scanapi OPENAPI_PATH SCANAPI_PATH 

Related issue: ADR 6: How to integrate feature that converts OpenAPI file to ScanAPI file

Description

Set the version on pyproject.toml file automatically. We want to avoid to manually bump the version for each release PR.

Maybe this would be a good candidate: https://github.com/mtkennerly/poetry-dynamic-versioning. It needs more investigation

Description

When a PR is opened and the CHANGELOG.md was not modified, send an alert to remember to update it.

Possible to be implemented with:

• Danger
• Peril

Or, alternatively, to create a CircleCI check that fails when it is not updated.

Description

Update CONTRIBUTING.md with:

• Instructions of how to create a release PR
• Instructions of how to deploy on docker hub with version tag

Create a Logo for ScanAPI

Some References: https://docs.google.com/document/d/1b8djucd09fjzzfGDsRYEyKa8c7VDZBgFVwJBr9Y52h8/edit?usp=sharing

A assertions.yaml with some assertions for each request.

for example, posts_list_all must have:

• x Length?
• this Structure?
• which type of data for each field?

Tasks:

• define a format for it
• implement how to run the assertions

in the future, we can create alerts when the assertions fails

Print request ID in the generated documentation. Something like:

### posts_list_all

GET  https://jsonplaceholder.typicode.com/posts

-> headers

Response: 200

-> headers

-> content

Given that ScanAPI already has a predefined yml structure, we could have a VSCode extension that autocompletes keys (endpoints, method, path, etc) based on where you are in the spec tree.

This would require further investigation to see what can be achieved but the following links should give some clarity:

• https://code.visualstudio.com/api/language-extensions/overview
• https://code.visualstudio.com/api/language-extensions/programmatic-language-features

Stop declaring the env vars on scanapi.yaml file and start getting these variables in fact from the env.

Description

ScanAPI report should be generated even if an Invalid Python Code error happens. It would help debugging.

Deploying to PyPI with GitHub Actions:
https://packaging.python.org/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/

Add coverage report tool: codecov

Description

Create a changelog file. More info at: https://keepachangelog.com/en/1.0.0/

Description

Add Black to Github Check or CI pipeline

Hi, I see that HTTP requests PATCH method are not implemented, for any particular reason?

Thank you.
Congratulations on the project, it is a great idea. 👌

To let the user know how log each response took:

https://stackoverflow.com/questions/43252542/how-to-measure-server-response-time-for-python-requests-post-request/43260678#43260678

• Add response time to Markdown default template
• Add response time to HTML default template

Description

Add the possibility to hide tokens and authorization info in the generated report to avoid expose sensitive information via configuration file (usually .scanapi.yaml).

Change the sensitive information value to <sensitive_information>

Configuration Options:

• report
  -- hide-response or hide-request
  --- headers or body or url
  ---- list of keys to hide

Example:

report:
  hide-response:
    headers:
      - Authorization
      - api-key
  hide-response:
    body:
      - api-key

The logic is implemented inside the hide_sensitive_info method

Example of how this should be rendered in the reports:

• header for request
• header for response
• body for request
• body for response
• url for request
• url for response

Dynamic and Static evaluations are mixed. Also we call evaluate from a lot of different parts of the code. This makes things hard to debug and to undestand

https://github.com/camilamaia/scanapi/blob/master/scanapi/variable_parser.py

https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L19-L26
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L32
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L50
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L25
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L36
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L56
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/root_node.py#L19
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/root_node.py#L31

Add a CI tool: https://circleci.com

https://circleci.com/setup-project/gh/camilamaia/scanapi

Description

ScanAPI should break when an Invalid Python Code error happens - after generating the report. It should summarise and show all the errors that happened.

https://github.com/pytest-dev/pytest/blob/83891d9022076375cede03bfd8c932d450e6fcf8/src/_pytest/config/__init__.py#L67

💄

Description

After the refactor on the reporter templates, hide sensitive info from headers is not working. It shows the real data instead of "<sensitive information>"

How to reproduce

Create a config file .scanapi.yaml with the content:

docs:
  hide:
    headers:
      - Authorization

Run scanapi with this config file and set an Authorization header inside the api specification:

api:
  base_url: ${BASE_URL}
  headers:
    Authorization: token123

The word token123 will appear in the report, instead of "<sensitive information>"

related to #15

• Add logs
• Change prints to logs
• Create a --debug option to more detailed logs too.

https://realpython.com/python-logging/
https://www.loggly.com/ultimate-guide/python-logging-basics/

To make console report prettier. And to add response time to it.

This is how it looks like now:

ScanAPI Report: Console
=======================

GET http://demo.scanapi.dev/api/health/ - 200
GET http://demo.scanapi.dev/api/languages/ - 200
GET http://demo.scanapi.dev/api/devs/ - 200
GET http://demo.scanapi.dev/api/devs/?newOpportunities=True - 200
GET http://demo.scanapi.dev/api/devs/?newOpportunities=False - 200
POST http://demo.scanapi.dev/api/devs/ - 201
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-51ad-9921-cea329bed7fa - 404
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0 - 200
DELETE http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0 - 200
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0/languages - 200

Maybe to add some different colours to each HTTP method?

💄