The operator testing can be run locally using the Ansible playbook at local-test-operator.yml. The playbook roles that run the operator tests used in this are identical to ones used as part of the CVP operator testing pipeline.

One notable exception is the package name uniqueness test which requires access to an internal database.

The playbook finishes successfully if all enabled tests pass. The logs and files for the test run can be found at /tmp/operator-test directory by default.

The testing playbooks has a number of prerequisites that it requires for a successful run. Most of these requirements need to be supplied as Ansible parameters when running the playbook.

The instructions for installing operator-courier can be found at this link.

The user needs to be currently logged into a working OCP cluster that will be used for testing as kubeadmin for the tests to work.

The path to the kubeconfig file for the OCP cluster needs to be supplied as the kubeconfig_path parameter, for example: -e kubeconfig_path=~/testing/kubeconfig

For rapid prototyping, you can spin up an OCP cluster using Red Hat CodeReady Workspaces

You can then specify your kubeconfig as follows: -e kubeconfig_path= ~/.crc/cache/crc_libvirt_4.2.14/kubeconfig

A valid quay.io namespace to which the user has access to needs to be supplied as the quay_namespace parameter, for example: -e quay_namespace="${QUAY_NAMESPACE}"

The testing process includes creating a private repository, be advised about the limits on the account owning the namespace.

The token for the quay.io account that owns the namespace used for testing can be obtained by using the following command:

QUAY_TOKEN=$(curl -sH "Content-Type: application/json" -XPOST https://quay.io/cnr/api/v1/users/login -d '
{
    "user": {
        "username": "'"${QUAY_USERNAME}"'",
        "password": "'"${QUAY_PASSWORD}"'"
    }
}' | jq -r '.token' | cut -d' ' -f2)

The token can then be supplied as the quay_token parameter, for example: -e quay_token="${QUAY_TOKEN}"

The operator metadata can be in either flattened or nested format - it will be copied and nested in a temporary directory for the purposes of the test.

The path to the directory containing the operator metadata can be supplied as the operator_dir parameter, for example: -e operator_dir=~/testing/operator-metadata

The rest of the required binaries are downloaded by the playbook to a temporary directory in /tmp/operator-test/bin and don't need to be installed manually.

If we, for some reason, want to skip the download (for example if we already have the required binaries at that location from a previous playbook run), we can set the run_prereqs parameter in this way: -e run_prereqs=false

The kube_objects is a kubernetes resource requires to be injected to the openshift cluster.

The kube_objects can be passed as a parameter to the playbook for example: -e kube_objects=kube_objects

If the kube_objects is a secret and in the encrypted form, a shared symmetric key is required to decrypt the kube_objects.

The symmetric_key can be passed as a parameter to the playbook for example: -e symmetric_key=symmetric_key

The path where the private key is stored can be supplied as rsa_private_key parameter, for example: -e rsa_private_key=~/.ssh/private_key

If we want to enable or disable individual tests, we can use these parameters and set them to true or false:

• run_lint for running operator-courier linting, default true
• run_catalog_init for running the catalog initialization test, default true
• run_deploy for deploying the operator to the testing cluster - this test is required for the subsequent tests, default true
• run_scorecard for running the operator scorecard tests on the operator that's deployed to the testing cluster, default true
• run_imagesource for checking the image sources of the tested operator - applies to Red Hat and ISV operators, otherwise disable with -e run_imagesource=false

The created resources and namespace are cleaned up after the playbook run by default. If we want to leave the resources after the run, we can set the run_cleanup parameter like this: -e run_cleanup=false

If we want to run the Red Hat operator tests, we invoke the playbook with the following command:

ansible-playbook -vv -i "localhost," --connection=local local-test-operator.yml \
    -e run_deploy=false \
    -e production_quay_namespace="redhat-operators" \
    -e operator_dir="${OPERATOR_DIR}"

Currently the access to an OCP cluster or an quay.io account is not required

If we want to run the full ISV operator testing, we invoke the playbook with the following command (inserting the aforementioned prerequisites):

ansible-playbook -vv -i "localhost," --connection=local local-test-operator.yml \
    -e kubeconfig_path="${KUBECONFIG_PATH}" \
    -e quay_token="${QUAY_TOKEN}" \
    -e quay_namespace="${QUAY_NAMESPACE}" \
    -e production_quay_namespace="certified-operators" \
    -e operator_dir="${OPERATOR_DIR}"

If we want to run full operator testing with image pull secrets and certified-operators

ansible-playbook -vv -i "localhost," --connection=local local-test-operator.yml \
    -e kubeconfig_path="${KUBECONFIG_PATH}" \
    -e quay_token="${QUAY_TOKEN}" \
    -e quay_namespace="${QUAY_NAMESPACE}" \
    -e production_quay_namespace="certified-operators" \
    -e operator_dir="${OPERATOR_DIR}" \
    -e kube_objects="${KUBE_OBJECTS}" \
    -e symmetric_key="${SYMMETRIC_KEY}" \
    -e rsa_private_key="${PRIVATE_KEY}"

If we want to run the operator testing for a community operator without running the imagesource test, we invoke the playbook with the following command:

ansible-playbook -vv -i "localhost," --connection=local local-test-operator.yml \
    -e kubeconfig_path="${KUBECONFIG_PATH}" \
    -e quay_token="${QUAY_TOKEN}" \
    -e quay_namespace="${QUAY_NAMESPACE}" \
    -e production_quay_namespace="community-operators" \
    -e operator_dir="${OPERATOR_DIR}" \
    -e run_imagesource=false