EPIC for QAMP fall 2021 Ecosystem project

Main task for 2021 fall QAMP ecosystem project will be rolling out support for community tier a.k.a projects developed by community.

Personas:

• community member of ecosystem
• developer/contributor of ecosystem repository
• core team member/curator

Stories:

• as community member of ecosystem I want to be able to submit my project so that it would be included into ecosystem
• as community member of ecosystem I want to be notified when my project is lacking behind stable / dev versions of Qiskit so that I would be able to update my codebase
• as community member of ecosystem I want my project to be displayed in ecosystem github README and qiskit.org
• as community member of ecosystem I want to receive membership badge so that it would demonstrate my ecosystem membership
• as developer/contributor of ecosystem repository I want to be able to propose new features for ecosystem
• as developer/contributor of ecosystem repository I want ecosystem management independent of platform so that it can be executed on any CI/CD framework
• as core team member/curator I want to be able to review submissions to ecosystem
• as core team member/curator I want CI processes takes care of tests, style checks, coverage after submission so that it would reduce time of review of submission
• as core team member/curator I want members of ecosystem tiers are stored as json serialized file within repository so that it can be used as reference database
• as core team member/curator I want README for repository to be generated automatically after each change of members
• as core team member/curator I want CI processes create PRs with members changes in case of successful tests/checks so that only thing remaining to include project into ecosystem would be merging PR
• as core team member/curator I want CI for tests/checks to be executed in timely manner (weekly) so that entire ecosystem always remains up-to-date
• as core team member/curator I want checks and tests to have Terra version associated with them so that we can track which version is breaking the code
• as core team member/curator I want track list of changes in members (diff between current and next version of members json)
• as core team member/curator I want badging system to be automated so that badges dynamically checks repository membership (ex: gray if checks did not passed or purple of checks passed)
• as core team member/curator I want ecosystem community tier to be separated into 3 subgroups: community member lvl 1, community member lvl 2 and candidate so that community member lvl 1 is project that checks all tests in guidelines (tests, coverage, code style), community member lvl 2 did not have enough coverage, candidate have tests or styles missing or failed

Ecosystem maintenance:

• tiers
  • main
  • member
  • candidate
  • custom tiers:
    • partners
    • research / universities
• main is curated
• candidate/members
  • if candidate passed all checks move it to members
  • if member did not passed tests move it to candidate
• separate workflows for main and custom tiers

Submission:

• template parser
• tests runs: stable, dev, linter
• add new entry to candidate tier if tests passed
• create PR to add to candidates

Badges system:

• members badge
• api backend for badges checks (https://github.com/marketplace/actions/dynamic-badges)

Steps to reproduce the problem

Run submission/maintanance workflow on repository
or

python manager.py python_dev_tests https://github.com/mickahell/qiskit-ecosystem_template --tier=COMMUNITY

What is the current behavior?

Dev terra tests fails with

2021-11-03 18:49:08 ecosystem INFO Traceback (most recent call last):
2021-11-03 18:49:08 ecosystem INFO File "<string>", line 1, in <module>
2021-11-03 18:49:08 ecosystem INFO AttributeError: module 'qiskit' has no attribute '__qiskit_version__'
2021-11-03 18:49:08 ecosystem INFO ERROR: InvocationError for command /home/runner/work/ecosystem/ecosystem/ecosystem/resources/cloned_repo_directory/qiskit-ecosystem_template/py39/bin/python -c 'import qiskit; f = open("./terra_version.txt", "w"); f.write(qiskit.__qiskit_version__["qiskit-terra"]); f.close();' (exited with code 1)
2021-11-03 18:49:08 ecosystem INFO ___________________________________ summary ____________________________________
2021-11-03 18:49:08 ecosystem INFO ERROR:   py39: commands failed

What is the expected behavior?

Should not fail on version request

What is the expected enhancement?

Right now messages are just plain tests. We probably need to make them a little bit more readable through modifying logger file.

Links:

• https://stackoverflow.com/questions/384076/how-can-i-color-python-logging-output
• https://pypi.org/project/coloredlogs/

What is the expected behavior?

Add ability to store project configuration in JSON Dao.

Order of configuration importance:

1. Config file
2. DB
3. Run default config

What is the expected behavior?

Submission for membership in ecosystem is handled through github issues. Specifically using submission template.
In order to make use of information inside issue we need to implement a parser, which will extract all necessary information.
Later on we will use this information in CI processes and save it to json db

A.k.a we need to implement function:

def parse_submission_issue(body_of_issue: str) -> Repository:
    """Parses body of issue and creates Repository object."""
    ...

Related story: as community member of ecosystem I want to be able to submit my project so that it would be included into ecosystem

Epic #9

What is the expected behavior?

Potential candidates to be added to ecosystem:

• https://github.com/CQCL/pytket-extensions - repository contains a collection of Python extension modules for CQC's pytket quantum SDK.
• https://github.com/QuTech-Delft/quantuminspire - platform allows to execute quantum algorithms using the cQASM language.
• https://github.com/unitaryfund/mitiq - Mitiq is a Python toolkit for implementing error mitigation techniques on quantum computers
• https://github.com/PennyLaneAI/pennylane-qiskit - The PennyLane-Qiskit plugin integrates the Qiskit quantum computing framework with PennyLane's quantum machine learning capabilities
• https://github.com/artificial-brain/quantumcat - quantumcat is a platform-independent, open-source, high-level quantum computing library, which allows the quantum community to focus on developing platform-independent quantum applications without much effort
• https://github.com/Blueqat/Blueqat - A quantum computing SDK
• https://github.com/qctrl/python-open-controls - Q-CTRL Open Controls is an open-source Python package that makes it easy to create and deploy established error-robust quantum control protocols from the open literature
• https://github.com/rigetti/qiskit-rigetti - Rigetti Provider for Qiskit
• https://github.com/acuto/quantum-dev - A multi-framework development environment for Quantum Computing on Kubernetes
• https://github.com/carstenblank/dc-qiskit-algorithms - Data Cybernetics qiskit-algorithms
• https://github.com/hennlo/Q-shall-not-pass - This game was built as part of the lecture: Quantum Information at the University of Basel in the Spring Semester of 2021
• https://github.com/infiniteregrets/QiskitBot - A discord bot that allows you to execute Quantum Circuits, look up the IBMQ Qiskit's Documentation, and search questions on the Quantum Computing StackExchange
• https://github.com/mit-han-lab/pytorch-quantum - A PyTorch-centric hybrid classical-quantum dynamic neural networks framework.
• https://github.com/qonwaygameoflife/qonwaygameoflife - Qiskit Camp - Hackaton Madrid 2019 - Quantum Game of Life
• https://github.com/MaxKelsen/quantumcircuit_gym - A custom gym environment for creating quantum circuits using IBMQ's qiskit library for python
• https://github.com/entropicalabs/polyadicQML - A python library to define, train and deploy Polyadic Quantum Machine Learning models
• https://github.com/dartmouth-cs98/Quantum-Tetris - Quantum Tetris is your traditional tetris game which has several twists based on the concepts of quantum computing.

What is the expected behavior?

1. In timely manner (weekly) CI should trigger workflow
2. Workflow should check all repositories in community tier with tests, coverage and style
3. If any repository does not follow guidelines move repository to list of candidates
4. Re-render readme and create PR with changes

Related stories:

• as core team member/curator I want README for repository to be generated automatically after each change of members
• as core team member/curator I want CI processes create PRs with members changes in case of successful tests/checks so that only thing remaining to include project into ecosystem would be merging PR
• as core team member/curator I want CI for tests/checks to be executed in timely manner (weekly) so that entire ecosystem always remains up-to-date

Epic: #9

What is the expected enhancement?

Automatically add the submission label to the Submission issue

EPIC #9

What is the expected enhancement?

Increase readability of docstrings for tests, classes and arguments.

What is the expected enhancement?

Improve contribution guide documentation and include all necessary steps for developer to setup env and start developing on ecosystem.

• env setup (pip install requirements)
• how to run tests
• how to perform style checks
• guidance on git (messages, branches, forking etc)

What is the expected behavior?

Implement function to run style checks for specified repository using specific linter configuration (from Qiskit-terra).

See python runner for getting the idea.

Runner should handle execution workflow for style checks and return summary

Related stories:

• as core team member/curator I want CI processes takes care of tests, style checks, coverage after submission so that it would reduce time of review of submission
• as core team member/curator I want checks and tests to have Terra version associated with them so that we can track which version is breaking the code
• as core team member/curator I want CI for tests/checks to be executed in timely manner (weekly) so that entire ecosystem always remains up-to-date

Epic #9

Steps to reproduce the problem

Run lint over a quantum circuit with measurement gate
ex : qc.measure(qram, c)

What is the current behavior?

E1101: Instance of 'QuantumCircuit' has no 'measure' member (no-member)

This error come from Terra

Resolve

We need to ignore this error in our Lint template

What is the expected enhancement?

Add workflows tests with act cmd inside the actual tests, lint, black, ... CI :

• Adapt workflows
• Adapt payload
• Adapt env
• Launch cmd tox -eci

What is the expected enhancement?

Write a notice to explain the requirements need to pass the submit form

What is the expected behavior?

Create Github template for python projects.

This is reserved issue to have logs of failed jobs.

Github repo

https://github.com/zlatko-minev/pyEPR

Description

Qiskit Metal E&M analysis with Ansys and the energy-participation-ratio method is based on pyEPR.

Email

[email protected]

Alternatives

No response

License

BSD 3-Clause "New" or "Revised" license

Affiliations

No response

Tags

plugin

What is the expected?

Implement function to run tests for specified repository against stable version of qiskit and against dev version (main branch).

See python runner for getting the idea.

Runner should handle execution workflow for tests and return summary

Related stories:

• as core team member/curator I want CI processes takes care of tests, style checks, coverage after submission so that it would reduce time of review of submission
• as core team member/curator I want checks and tests to have Terra version associated with them so that we can track which version is breaking the code
• as core team member/curator I want CI for tests/checks to be executed in timely manner (weekly) so that entire ecosystem always remains up-to-date

Epic #9

Github repo

https://github.com/mickahell/quantum_pokemon-fight

Description

...

Email

[email protected]

Alternatives

No response

License

MIT

Affiliations

No response

Tags

tool

Structure (3 minutes):

• why? - 30 seconds explanation:
  • promote community projects
  • keep ecosystem stable
• how? - 45 seconds
  • submission wf
  • weekly
  • badge
• recorded videos: 1:15 seconds -
  • submission wf
    • show your project
    • go to ecosystem page
    • click join ecosystem -> fill form -> submit
    • show actions
    • show comment and PR
  • optional: quickly show weekly action (old one for 10 seconds)
  • optional: show badges
  • show ecosystem on qiskit.org POC
• recap: 30 seconds
  • what we have planned
  • what we have been achieved

Github repo

https://github.com/IceKhan13/demo-implementation

Description

Some change here

Email

[email protected]

Alternatives

No response

License

Apache License 2.0

Affiliations

No response

Tags

tool, tutorial

Task

Create template and parser for configuration to allow some flexibility on configuring how to run tests, style checks and coverage.

What is the expected behavior?

Configuration should support:

• language (default: Python)
• what packages to install
• requirements files
• how to run tests
• how to run styles check
• how to run coverage

Steps to reproduce the problem

Launch the job stable Tests stable check_
Link of the run : https://github.com/qiskit-community/ecosystem/runs/4098609837?check_suite_focus=true

What is the current behavior?

2021-11-03 21:56:51 ecosystem WARNING Test result was not saved.There is not repo for url https://github.com/mickahell/qiskit-ecosystem_template
2021-11-03 21:56:51 ecosystem INFO Test results for https://github.com/mickahell/qiskit-ecosystem_template: TestResult(False, STABLE_COMPATIBLE, 0.18.3)
2021-11-03 21:56:51 ecosystem INFO Setting output variable PASS: False - Terra version : 0.18.3
0.18.3

The WARNING looks to make the check at False.

What is the expected behavior?

Deprecation warning for aqua should not fail tests

What is the expected enhancement?

Implement CI for tests, linter, black, coverage similar to Terra

2 directories :

• /tests/resources
• /tests/ressources

Epic #9

What is the expected enhancement?

Skip dev and stable tests if there is no dependency on Qiskit

Steps to reproduce the problem

• run dev test script for repository that has qiskit metapackage in requirements

What is the current behavior?

When dev workflow tries to install qiskit-terra separately pip fails due to version inconsistancy with metapackage.

2021-12-06 15:29:40 ecosystem INFO Successfully installed pbr-5.8.0 qiskit-terra-0.19.0 stevedore-3.5.0
2021-12-06 15:29:40 ecosystem INFO py39 run-test: commands[2] | python -c 'import qiskit; f = open("./terra_version.txt", "w"); f.write(qiskit.__qiskit_version__["qiskit-terra"]); f.close();'
2021-12-06 15:29:42 ecosystem INFO py39 run-test: commands[3] | pip check
2021-12-06 15:29:43 ecosystem INFO qiskit 0.32.1 has requirement qiskit-terra==0.18.3, but you have qiskit-terra 0.19.0.
2021-12-06 15:29:43 ecosystem INFO ERROR: InvocationError for command /home/runner/work/ecosystem/ecosystem/ecosystem/resources/cloned_repo_directory/qiskit-superstaq/py39/bin/pip check (exited with code 1)

What is the expected behavior?

Should not fail

Steps to reproduce the problem

Create PR

What is the current behavior?

PRs does not trigger github actions

What is the expected behavior?

CI should be triggered for each PR and push

What is the expected enhancement?

Print depreciation warning in the tests check report in the submission issue

Epic #9

Steps to reproduce the problem

python manager.py python_coverage  https://github.com/IceKhan13/demo-implementation --tier="MAIN" --coverage_type="whatever"

What is the current behavior?

Does not perform coverage checks

INFO:root:Cloning repository: https://github.com/IceKhan13/demo-implementation
INFO:root:b''
INFO:root:RETURN CODE: 0}
INFO:root:Tox file exists.
INFO:root:b'coverage create: /Users/iskandarsitdikov/Documents/development/work/ecosystem/ecosystem/resources/cloned_repo_directory/demo-implementation/coverage\n'
INFO:root:b'coverage installdeps: pytest, -rrequirements.txt\n'
INFO:root:b'coverage develop-inst: /Users/iskandarsitdikov/Documents/development/work/ecosystem/ecosystem/resources/cloned_repo_directory/demo-implementation\n'
INFO:root:b'coverage installed: attrs==21.2.0,-e git+https://github.com/IceKhan13/demo-implementation@a142c0c0dbd5b331ca55f937e165da8334143f82#egg=demo_impl,iniconfig==1.1.1,packaging==21.0,pluggy==0.13.1,py==1.10.0,pyparsing==2.4.7,pytest==6.2.4,toml==0.10.2\n'
INFO:root:b"coverage run-test-pre: PYTHONHASHSEED='3260761479'\n"
INFO:root:b'coverage run-test: commands[0] | coverage3 -m pytest\n'
INFO:root:b'WARNING: test command found but not installed in testenv\n'
INFO:root:b'  cmd: /opt/anaconda3/envs/ecosystem/bin/coverage3\n'
INFO:root:b'  env: /Users/iskandarsitdikov/Documents/development/work/ecosystem/ecosystem/resources/cloned_repo_directory/demo-implementation/coverage\n'
INFO:root:b'Maybe you forgot to specify a dependency? See also the allowlist_externals envconfig setting.\n'
INFO:root:b'\n'
INFO:root:b'DEPRECATION WARNING: this will be an error in tox 4 and above!\n'
INFO:root:b'no such option: -m\n'
INFO:root:b"Use 'coverage3 help' for help.\n"
INFO:root:b'Full documentation is at https://coverage.readthedocs.io\n'
INFO:root:b'ERROR: InvocationError for command /opt/anaconda3/envs/ecosystem/bin/coverage3 -m pytest (exited with code 1)\n'
INFO:root:b'___________________________________ summary ____________________________________\n'
INFO:root:b'ERROR:   coverage: commands failed\n'
INFO:root:b''
INFO:root:RETURN CODE: 1}

What is the expected behavior?

Should attempt to do coverage tests

What is the expected behavior?

Create parametric reusable Github action to run tests.

Those actions will be used in workflows for submission and weekly checks.

What is the expected enhancement?

In #116 we added support for storing configurations for projects in DB.
Now we need to update members.json file with configuration for projects.

Example:
https://github.com/qiskit-community/ecosystem/blob/110b47a30c6821d08ddd472939715aa20c6f495b/ecosystem/resources/members.json#L467-L483

Steps to reproduce the problem

Create a new issue from the submission from or tape "!redo" as comment in a submission issue

What is the current behavior?

• The outputs from tests check seems not be interpreter
• The issue commenting seems to need some access
• The gloval env vars seems not to be present this way

Link of the run : https://github.com/mickahell/ecosystem/runs/3968319720?check_suite_focus=true

What is the expected behavior?

• Having those var fill
• Be able to comment issue

Epic #9

What is the expected behavior?

Implement function to run coverage tests for specified repository against stable version of qiskit.

See python runner for getting the idea.

Runner should handle execution workflow for coverage and return summary

Related stories:

• as core team member/curator I want CI processes takes care of tests, style checks, coverage after submission so that it would reduce time of review of submission
• as core team member/curator I want checks and tests to have Terra version associated with them so that we can track which version is breaking the code
• as core team member/curator I want CI for tests/checks to be executed in timely manner (weekly) so that entire ecosystem always remains up-to-date

Epic #9

Summary

Allow the submission of no python code, it can be :

• docker template image,
• project using micro-qiskit

What is the expected enhancement?

Currently anyone can type !redo and spam our CI with job requests.
We need to restrict this possibility to only authors and admins (members of community) of repo

Github repo

https://github.com/mickahell/qiskit-ecosystem_template

Description

My awesome project !

Email

[email protected]

Alternatives

No response

License

Apache License 2.0

Affiliations

Me, my and myself

Tags

paper-implementation

What is the expected behavior?

Add CRUD though python cli

• add repo
• move repo to other tier
• get all repos
• add version of terra to repo

What is the expected behavior?

Add language support for configuration

For example: for Python it would be 3.6, 3.7 etc

{
     "language": {
           "name": "python",
           "versions": ["3.6", "3.7"]
     }
}

Github repo

https://github.com/mickahell/qiskit-ecosystem_template

Description

Project template for adding to Qiskit Ecosystem

Email

[email protected]

Alternatives

No response

License

Apache License 2.0

Affiliations

Qiskit community

Tags

tool

What is the expected behavior?

Implement badging system for members of ecosystem.

Badging should be dynamic and reflect state of json db with members.

Potential solution: https://github.com/marketplace/actions/dynamic-badges

Readme should have collapsable section for each of member of ecosystem which has code of badge, so developers can add this code in their readme to showcase status of membership.

Later on we will implement CI to submit badging for members repositories.

Related stories: as core team member/curator I want badging system to be automated so that badges dynamically checks repository membership (ex: gray if checks did not passed or purple of checks passed)

Epic: #9

Steps to reproduce the problem

• create a submission issue or use #21
• type !redo in comments
• go to https://github.com/qiskit-community/ecosystem/actions

What is the current behavior?

Failing on !redo

What is the expected behavior?

Should work on comment !redo.
Parser should work.

Need to write test cases with all variety of extra spaces, tabs, new line etc.

What is the expected enhancement?

• Improve readme template for better readability.
• Incorporate tests results in descriptions.
• Split into template hierarchy. See template inheritance

Related stories:

• as community member of ecosystem I want my project to be displayed in ecosystem github README and qiskit.org
• as core team member/curator I want README for repository to be generated automatically after each change of members

Epic #9

What is the expected behavior?

Each time changes are happening in json db we need to store list of changes for future reference.

List of changes should be also json file and controlled by changes controller.

Related stories: as core team member/curator I want track list of changes in members (diff between current and next version of members json)

Epic: #9

Steps to reproduce the problem

Run batch repo check

What is the current behavior?

Due to autostash and concurrent runs of check jobs per each repository workflow is pushing conflicting/stashed commits to batch PR.

Example: https://github.com/qiskit-community/ecosystem/pull/105/files#diff-822e9a536085a1bf3ca0dc06cd53601d2c444f758935a5903d9852ec49aedaa5R591

What is the expected behavior?

Batch checks should not create conflicting PRs.

Suggestions

One way to resolve this would be: instead of saving results into the same file save tests results as separate files for each project. And create new actions to read those files and push to db in a single commit on post PR merge or on specific comment / tag

What is the expected behavior?

1. Creation of submission should trigger workflows to run tests, coverage and styles against submitted repository.
2. Workflow run tests, coverage and styles against submitted repository.
3. If something is incorrect CI should comment on issue about problems.
4. When problems will be fixed in submitted repository author should be able to comment on submission issue command /resubmit to restart CI workflow on step 2.
5. If everything is good CI should add repository to community members tier in json db, rerender README and create PR with changes against main branch.

Related stories:

• as core team member/curator I want CI processes takes care of tests, style checks, coverage after submission so that it would reduce time of review of submission

Epic: #9

What is the expected behavior?

• Create short description of progress made between checkpoint 1 and November.
• Upload pictures on project progress
  • Abby's POC for qiskit.org ecosystem page
  • Screenshots of github actions workflows
  • Workflow diagram

What is the expected behavior?

Review ecosystem configuration files.

Currently configuration supports extra dependencies and tests commands and only works for python repositories and using tox as testing and configuration (which is good actually and follows other Qiskit projects)

We need at least to add extra configurable commands to run style checks and coverage.

Epic: #9

What is the expected

We need to have 2 projects ready to be add in the ecosystem.
Can be our own project or from other advocates

Requirements

• using qiskit
• be compatible qiskit latest and qiskit dev (master branch)
• having unit tests (>80% coverage)
• having a well linter (could use black)

Epic #9

What is the expected enhancement?

Add a json lint in the CI, to check the good syntax of members.json