icekhan13 / purplecaffeine Goto Github PK

Implement Jupyter widget to explore saved trials.

from purple_caffeine import Widget, LocalBackend
Widget.render(LocalBackend(file_storage_path="./"))

Issues:

• #6
• #7

What is the expected enhancement?

Implement search functionality. Let's start with search by title and description.

Details

In client:

backend.list(query="<my-seach-term>")

API backend: /trials/?query=<bla-bla-my-search-term>

Trial.objects.filter(title__contains="<my_search_term>" or description__contain="<my_search_term>")

LocalBackend: TBD

We need high level design on widget.
It can be simple sketch on a piece of paper, but we need all know how it will be looking like at the end.

Epic #3

Summary

Dockeurize the api, that can be running using flags.

Example of api_server/docker/Dockerfile :

FROM ubuntu:latest

ARG DEBIAN_FRONTEND=noninteractive
ARG  PROJECT_NAME=purplecaffeine

LABEL org.opencontainers.image.title="${PROJECT_NAME}" \
      org.opencontainers.image.authors=${GITHUB_ACTOR} \
      org.opencontainers.image.vendor=${GITHUB_REPOSITORY} \
      org.opencontainers.image.source="${PROJECT_URL}" \
      org.opencontainers.image.url="${PROJECT_URL}/tags" \
      org.opencontainers.image.description="API purplecaffeine" \
      org.opencontainers.image.os="ubuntu" \
      org.opencontainers.image.version=${GITHUB_REF}

COPY . /opt/purplecaffeine

RUN apt-get update -yq \
&& apt-get install python3-pip -y

RUN pip3 install --upgrade pip setuptools
RUN pip3 install -r /opt/purplecaffeine/requirements.txt

WORKDIR /opt/purplecaffeine

VOLUME /etc/purplecaffeine/conf/
EXPOSE 8080

ENTRYPOINT [ "/opt/purplecaffeine" ]
CMD [ "--help" ]

And so that can be run by calling from api_server/ :

docker run \
		--name purplecaffeine \
		-v $(PWD)/test:/etc/purplecaffeine/conf \
		-e PASS_DB=${PASS_DB} -e TOKEN_API=${TOKEN_API} \
		-p 8080:8080 \
		purplecaffeine:latest \
		-conf=/etc/purplecaffeine/conf/conf_docker.yaml \
		-swagger=/etc/purplecaffeine/conf/swagger.yaml

Or something similar.

Prepare submission structure for repository for JoSS https://joss.theoj.org/

Instructions https://joss.readthedocs.io/en/latest/submitting.html

What is the expected enhancement?

Add tags to Trial class.

What is the expected enhancement?

Currently requirements for project is set as ==. We need to update it to >=

https://github.com/IceKhan13/purplecaffeine/blob/main/client/requirements.txt#L2

Summary

• Rename backend folder as api_server in order to be clearer.
• Added some documentation to start explaining which blocks.

Summary

Create the CI with GitHub to automate the build of the docker image and the push into our GitHub repo (or later DockerHub).

Example of CI :

jobs:
  docker:
    name: Build purplecaffeine
    runs-on: ubuntu-latest
    permissions:
      packages: write
      contents: read
    env:
      IMAGE_NAME: "purplecaffeine_server"
      BASEFOLDER: "api_server"
    steps:
    - uses: actions/checkout@v2
    - name: Build image
      run: |
        cd ${{env.BASEFOLDER}}
        docker build . --file docker/Dockerfile --tag ${{ env.IMAGE_NAME }}:latest
    - name: Log in to registry
      run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
    - name: Push image
      run: |
        IMAGE_ID=ghcr.io/${{ github.repository_owner }}/${{ env.IMAGE_NAME }}
        IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
        docker tag ${{ env.IMAGE_NAME }} $IMAGE_ID:latest
        docker push $IMAGE_ID:latest

Or something similar.

Steps to reproduce the problem

backend = LocalBackend("<...>")
backend.get("bla-bla") # non existing name

What is the current behavior?

What is the expected behavior?

should log warning and return None or something :)

What is the expected enhancement?

Use the new Qiskit sphinx theme for the doc template.
See : Qiskit-Extensions/circuit-knitting-toolbox#313

Implement client for PurpleCaffeine.

This includes Trial class and LocalBackend.

Methods for Trial class

def add_metric(self, name: str, value: float):
   ...
def add_parameter(self, name: str, value: str):
   ...
def add_circuit(self, name: str, circuit: QuantumCircuit):
   ...
def add_backend(self, name: str, backend: Backend):
   ...
def add_operator(self, name: str, operator: Operator):
   ...
def add_artifact(self, name: str, path_to_file: str):
   ...

LocalBackend should save and retrieve data to/from given folder.

Purposed format to store data is json as it is human readable.

Issue in epic:

• #8
• #9
• #11
• #22
• #24

What is the expected enhancement?

For running a unit test of the remote backend of client, we need to setup the API in the Actions.

Todo

• Setup postgres DB
• Setup API

What is the expected enhancement?

Create jupyter tutorial on how to track building algorithm using :

• Basic guide
• Guide how to use the API
• QAOA
• VQE
• QUBO

What is the expected enhancement?

• Make the submission into the Qiskit Ecosystem : Submission form
• Add badge into readme

Implement restful api for APIBackend to store Trials in remote application.

Epic #2

What is the expected behavior?

Integrate a swagger to have an interactive API.

Summary

Create a basic CI for automation using tox.

Details

• Launch from CI
• Commands in tox
• Unit tests
• Lint
• black
• Coverage
• ecosystem check

What is the expected enhancement?

Create a How to documentation, explaining :

• how to use the client with/out the widget
• how to use the api from command line
• how to install the api in local

Make this doc appear on the sphinx doc.

What is the expected enhancement?

Add publication badge to READme and doc

[![DOI](https://joss.theoj.org/papers/10.21105/joss.05673/status.svg)](https://doi.org/10.21105/joss.05673)

What is the expected enhancement?

Add endpoint tests for API service.

Ref: https://www.django-rest-framework.org/api-guide/testing/#api-test-cases

Required to address JoSS review comments openjournals/joss-reviews#5673 (comment)

What is the expected behavior?

Add default value in tox variable call.

from {env:FOLDER:} --> {env:FOLDER:client}

What is the expected enhancement?

In order to smoothly integrate purple caffeine with other system it might be beneficial to be able to create object without exactly specifing all constructor parameters, but provide them though env variables.

Let me explain. Instead of this

class Trial:
  def __init__(self, name: str, storage: Optional[BaseStorage] = None):
    self.name = name
    self.storage = storage

have something like this

class Trial:
  def __init__(self, name: Optional[str] = None, storage: Optional[BaseStorage] = None):
    self.name = name or os.environ.get("PURPLE_CAFFEINE_TRIAL_NAME")
    if self.name is None:
      raise PurplecaffeineException("Please specify name or trial or configure it using env variables")
    if storage is None:
      storage_type = os.environ.get("PURPLE_CAFFEINE_STORAGE_CLASS", "LocalStorage")
      storage_mapping: Dict[str, BaseStorage] = {
        "LocalStorage": LocalStorage,
        "S3Storage": S3Storage,
        "ApiStorage": ApiStorage
      }
      self.storage = storage_type.get(storage_type)()
    else:
      self.storage = storage

Now if we have all env variable properly set we can just write

with Trial() as trial:
    trial.add_...

We need to have the same behavior with all parameters for Trial, LocalStorage, S3Storage, ApiStorage.

What is the expected enhancement?

Users might be confused by another Backend object, which overlaps with qiskit.Backend

What is the expected enhancement?

Update client API unit test in order to generate docker container from test.

See : https://testcontainers-python.readthedocs.io/en/latest/README.html

Initial implementation of a view of a widget.

Epic #3

Docs: build documentation for purplecaffeine using sphinx and github pages to deploy them

What is the expected behavior?

Having a function to import and export Trial in order to facilitate the share of experiments.

• import
• export

What is the expected enhancement?

Add code of conduct to secure all repository checks

What is the expected enhancement?

Add installation instruction for client folder

Steps to reproduce the problem

Add multiples circuits or multiples graphs and show them with the widget.

What is the current behavior?

The window is not height enough.

What is the expected behavior?

Having a window with a 100% height

What is the expected enhancement?

In order to avoid having a HUGE file, we need to split the different element of the trial.json file into different files.

/path
- /trial_<uuid>
-- /trial.json # trial name, id, tags, description, etc
-- /circuit_1.json # circuit 1
-- /circuit_2.json # circuit 2
-- /circuit_3.json # circuit 3
-- /text_1.json # text 1

What is the expected behavior?

Create unit test for the API.

What is the expected enhancement?

Implement the save and read backend of Trial class.

Steps to reproduce the problem

It seems, we can't access the data of the API from the browser.

What is the expected enhancement?

Currently in the sphinx documentation, the properties appear like a simple very big block without any format :

Properties:
metrics: list of metric, like number of qubits parameters: list of parameter, like env details circuits: list of quantum circuit qbackends: list of quantum backend operators: list of operator, like Pauli operators artifacts: list of artifact path, any external files path texts: list of text, any descriptions arrays: list of array, like quantum circuit results

• We need to find a way to make it appear in a format way.
• Fix any Sphinx CI problem

What is the expected enhancement?

Currently last cell in tutorial has this content:

# list all files in ./trials folder
trials_files = [x.split('.')[0] for x in (os.listdir('./trials'))]
# read the first one
local_backend.get(trials_files[0])
# same as local_backend.get("88b5ef54-a66b-45e1-aed8-c7db7ad82594")

which is very confusing for non developer audience, which is our target audience.

Instead of listing file in directory just get list of trials and select first one

trials = backend.list()

random_trial = trials[0]
trial_uuid = random_trial.uuid

backend.get(trial_uuid)

Implement server side API service to store and retrieve data of Trials.

Issues in epic:

• #11
• #10

What is the expected behavior?

Add an authentication system for requesting the remote API.

Questions

• How do we want to manage the authentication with the remote API ?
  Using the IBMQ experience like Qiskit does or something else ... ?
• Do we want to do that for the QAMP or for after ?

Implement LocalBackend which will store trials on local disk in form of files.

Epic #1

Implement Trial class.

Methods for Trial class

def add_metric(self, name: str, value: float):
   ...
def add_parameter(self, name: str, value: str):
   ...
def add_circuit(self, name: str, circuit: QuantumCircuit):
   ...
def add_backend(self, name: str, backend: Backend):
   ...
def add_operator(self, name: str, operator: Operator):
   ...
def add_artifact(self, name: str, path_to_file: str):
   ...
def add_text(self, text: str):
   ...
def add_array(self, array: Union[np.ndarray, List[Any]):
    ...

Epic #1

What is the expected enhancement?

Adapt the client to add token call, and description for the API backend.

• Token call
• Call API with token
• Add description to Trial
• Not using Configuration host anymore
• Corrected the image call in the README :

![Logo](https://raw.githubusercontent.com/IceKhan13/purplecaffeine/main/docs/images/readme_logo.png)

What is the expected enhancement?

Implement action to setup PyPi release procedure on manual trigger (for now)

What is the expected enhancement?

Write a minimal documentation of all the feature about the client.

What is the expected behavior?

Implement s3 backend to store trials in s3 bucket.

Details

Client: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html

Example

from purplecaffeine import S3Backend

s3_backend = S3Backend(
   bucket_name="my_trials",
   key="<API_KEY>",  # this can be env
   access_key="<API_ACCESS_KEY>" # this can be env
)

Implement ApiBackend which will be backend for storing trials data.

Epic #2

What is the expected behavior?

Add the qiskit version into Trial object.

What is the expected behavior?

Implement the search in the localbackend.

What is the expected enhancement?

Sometimes the circuit can be very large so can cause latency and problem in the widget.
We want to add a button to allow to show the circuits.

Default : hidden
onclick : showing

What is the expected enhancement?

Create a CI in order to have unittests for the jupyter tutorial.

Example of how to do that :

    - name: Test notebook
      run: |
        python -m pip install jupyter nbconvert nbformat
        jupyter nbconvert --to notebook --execute docs/guides/*.ipynb --inplace