scanapi / scanapi Goto Github PK

Create a README header, something like:

https://github.com/barbosa/clorox

Some References: https://docs.google.com/document/d/1b8djucd09fjzzfGDsRYEyKa8c7VDZBgFVwJBr9Y52h8/edit?usp=sharing

💄

Dynamic and Static evaluations are mixed. Also we call evaluate from a lot of different parts of the code. This makes things hard to debug and to undestand

https://github.com/camilamaia/scanapi/blob/master/scanapi/variable_parser.py

https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L19-L26
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L32
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/request_node.py#L50
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L25
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L36
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/endpoint_node.py#L56
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/root_node.py#L19
https://github.com/camilamaia/scanapi/blob/master/scanapi/tree/root_node.py#L31

Enable to have API spec in multiples files using an include syntax.

To let the user know how log each response took:

https://stackoverflow.com/questions/43252542/how-to-measure-server-response-time-for-python-requests-post-request/43260678#43260678

• Add response time to Markdown default template
• Add response time to HTML default template

Description

ScanAPI report should be generated even if an Invalid Python Code error happens. It would help debugging.

Current, when there is an error when executing a command inside the python code tag ${{ }}, this is the message:

ERROR:scanapi.evaluators.string_evaluator:Invalid Python code defined in the API spec: Expecting value: line 1 column 1 (char 0)

We should improve it to show precisely which error happened.

Create a Logo for ScanAPI

Some References: https://docs.google.com/document/d/1b8djucd09fjzzfGDsRYEyKa8c7VDZBgFVwJBr9Y52h8/edit?usp=sharing

Description

Error when running scanapi v0.0.16

jinja2.exceptions.TemplateNotFound: markdown.jinja```

Generate automated code documentation from the docstrings.

https://wiki.python.org/moin/DocumentationTools
https://www.sphinx-doc.org/en/master/

Something similar to the right column in the post docs:

Starting with curl is good enough 👍

Add a CI tool: https://circleci.com

https://circleci.com/setup-project/gh/camilamaia/scanapi

Description

When a PR is opened and the CHANGELOG.md was not modified, send an alert to remember to update it.

Possible to be implemented with:

• Danger
• Peril

Or, alternatively, to create a CircleCI check that fails when it is not updated.

Create ScanAPI spec from an OpenAPI specification

Create a command that converts an OpenAPI specification file into a ScanAPI specification file.
OpenAPI Specification: https://swagger.io/specification/

For that, we need to create a new Click command called convert in the __main__.py file.

With the arguments:

• input file path (required)
• output file path (required)

And with the options:

• -f, --from, input_format. (default should be openapi, required)
• -t, --to, output_format. (default should be scanapi, required)

Example:

$ scanapi convert -f openapi -t scanapi OPENAPI_PATH SCANAPI_PATH 

Related issue: ADR 6: How to integrate feature that converts OpenAPI file to ScanAPI file

To make console report prettier. And to add response time to it.

This is how it looks like now:

ScanAPI Report: Console
=======================

GET http://demo.scanapi.dev/api/health/ - 200
GET http://demo.scanapi.dev/api/languages/ - 200
GET http://demo.scanapi.dev/api/devs/ - 200
GET http://demo.scanapi.dev/api/devs/?newOpportunities=True - 200
GET http://demo.scanapi.dev/api/devs/?newOpportunities=False - 200
POST http://demo.scanapi.dev/api/devs/ - 201
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-51ad-9921-cea329bed7fa - 404
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0 - 200
DELETE http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0 - 200
GET http://demo.scanapi.dev/api/devs/129e8cb2-d19c-41ad-9921-cea329bed7f0/languages - 200

Maybe to add some different colours to each HTTP method?

💄

Error when running scanapi v0.0.15

    from scanapi.tree.api_tree import APITree
ModuleNotFoundError: No module named 'scanapi.tree'

Description

Make coverage > 90% for each file. You can find the current % for each file here: https://codecov.io/gh/scanapi/scanapi/tree/master/scanapi

Currently we are evaluating python code from API specification ${{ code }} using eval:

We must use a safer and more elegant solution. I am out of ideas here.

• Templates (like jinja) don't work because we need to evaluate dynamically the python code while the requests are being made. The python code might depend on the last request response, for example.
• ast.literal_eval does not support access to external variables

Stop declaring the env vars on scanapi.yaml file and start getting these variables in fact from the env.

Hi, I see that HTTP requests PATCH method are not implemented, for any particular reason?

Thank you.
Congratulations on the project, it is a great idea. 👌

Current the format is markdown.

Description

At PyPi page desccription the image scanapi-report-example.png is not loading

This happens because it is being used the relative path to link the image on README.md

How it is now:

<p align="center">
  <img src="images/scanapi-report-example.png" width="700">
</p>

How it is should be:

<p align="center">
  <img src="https://github.com/camilamaia/scanapi/blob/master/images/scanapi-report-example.png" width="700">
</p>

Description

Create a changelog file. More info at: https://keepachangelog.com/en/1.0.0/

Given that ScanAPI already has a predefined yml structure, we could have a VSCode extension that autocompletes keys (endpoints, method, path, etc) based on where you are in the spec tree.

This would require further investigation to see what can be achieved but the following links should give some clarity:

• https://code.visualstudio.com/api/language-extensions/overview
• https://code.visualstudio.com/api/language-extensions/programmatic-language-features

Show Request Body, if it exists, on Markdown Report. Current it is showing only response body.

Description

We need to ensure that API spec has some mandatories keys in order to work properly.

The first mandatory key that need to be checked is the key api. The specification should start with it.
https://github.com/scanapi/scanapi/blob/master/scanapi/__init__.py#L68

Under the key endpoints, we need to ensure each entry has at least a name and a requests key
https://github.com/scanapi/scanapi/blob/master/scanapi/tree/endpoint_node.py#L79

Under the key requests, we need to ensure each entry has at least a name and a path key.
https://github.com/scanapi/scanapi/blob/master/scanapi/tree/request_node.py#L106

This is an example of a minimal possible structure:

api:
  endpoints:
    - name: scanapi-demo
      requests:
        - name: health
          path: http://demo.scanapi.dev/api/health/

If any of this mandatories keys is missing, an error should be raised.

Print request ID in the generated documentation. Something like:

### posts_list_all

GET  https://jsonplaceholder.typicode.com/posts

-> headers

Response: 200

-> headers

-> content

Description

Update CONTRIBUTING.md with:

• Instructions of how to create a release PR
• Instructions of how to deploy on docker hub with version tag

Description

After the refactor on the reporter templates, hide sensitive info from headers is not working. It shows the real data instead of "<sensitive information>"

How to reproduce

Create a config file .scanapi.yaml with the content:

docs:
  hide:
    headers:
      - Authorization

Run scanapi with this config file and set an Authorization header inside the api specification:

api:
  base_url: ${BASE_URL}
  headers:
    Authorization: token123

The word token123 will appear in the report, instead of "<sensitive information>"

Description

Configure a changelog linter. Maybe https://github.com/rcmachado/changelog

Related with: #88

Problem

If we define a var containing an upper case letter, like:

vars:
    apiKey: abc123

and we try to use it later as ${apiKey} it will raise the error:

ERROR:scanapi.evaluators.string_evaluator:'apiKey' environment variable not set or badly c

Possible Solution

To check if there is any lower case in the word here:

https://github.com/camilamaia/scanapi/blob/master/scanapi/evaluators/string_evaluator.py#L48

if any(letter.islower() for letter in variable_name):
  continue

A assertions.yaml with some assertions for each request.

for example, posts_list_all must have:

• x Length?
• this Structure?
• which type of data for each field?

Tasks:

• define a format for it
• implement how to run the assertions

in the future, we can create alerts when the assertions fails

Description

Add Black to Github Check or CI pipeline

Description

ScanAPI should break when an Invalid Python Code error happens - after generating the report. It should summarise and show all the errors that happened.

https://github.com/pytest-dev/pytest/blob/83891d9022076375cede03bfd8c932d450e6fcf8/src/_pytest/config/__init__.py#L67

Create a Makefile with the following commands:

• Install
• Test

Description

Add the possibility to hide tokens and authorization info in the generated report to avoid expose sensitive information via configuration file (usually .scanapi.yaml).

Change the sensitive information value to <sensitive_information>

Configuration Options:

• report
  -- hide-response or hide-request
  --- headers or body or url
  ---- list of keys to hide

Example:

report:
  hide-response:
    headers:
      - Authorization
      - api-key
  hide-response:
    body:
      - api-key

The logic is implemented inside the hide_sensitive_info method

Example of how this should be rendered in the reports:

• header for request
• header for response
• body for request
• body for response
• url for request
• url for response

Deploying to PyPI with GitHub Actions:
https://packaging.python.org/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/

factory_boy is a fixtures replacement based on thoughtbot’s factory_bot

As a fixtures replacement tool, it aims to replace static, hard to maintain fixtures with easy-to-use factories for complex object.

Instead of building an exhaustive test setup with every possible combination of corner cases, factory_boy allows you to use objects customized for the current test, while only declaring the test-specific fields:

💄

• Show only path_url in the titles
• Add response and request headers
• Add request headers and body
• Add response headers, metada and body
• Add cURL
• Add response time

• Add logs
• Change prints to logs
• Create a --debug option to more detailed logs too.

https://realpython.com/python-logging/
https://www.loggly.com/ultimate-guide/python-logging-basics/

Similar to headers implementation

api:
  base_url: ${BASE_URL}
  headers:
    Authorization: ${BEARER_TOKEN}
  params:
    per_page: 10

related to #15

Add coverage report tool: codecov

Description

Set the version on pyproject.toml file automatically. We want to avoid to manually bump the version for each release PR.

Maybe this would be a good candidate: https://github.com/mtkennerly/poetry-dynamic-versioning. It needs more investigation