Updated custom namespace in 2 locations as below.

1. conf/ocsci/default_config.yaml ==> ROOK_CLUSTER_NAMESPACE: "rook-ceph"
2. ocs/defaults.py ==> ROOK_CLUSTER_NAMESPACE = 'rook-ceph'

After editing in above 2 places still my oc commands are generated with "openshift-storage" namespace.

This param should be overridden from one location.

Workaround: This is not a good suggestion but works for executing tests.
defaults.ROOK_CLUSTER_NAMESPACE = 'rook-ceph' ==> in the test_case.py

`self = <tests.test_ocs_260.TestCaseOCS260 object at 0x7f7216e1f048>

def test_run_ocs_260(self):
    """
    Test case OCS-260

    """
    pvc_count = 4
    pv_list = []

  create_ceph_block_pool()

tests/test_ocs_260.py:400:

tests/test_ocs_260.py:103: in create_ceph_block_pool
assert CBP.create(yaml_file=TEMP_YAML)
../../ocs-ci/ocs/ocp.py:119: in create
return self.exec_oc_cmd(command)
../../ocs-ci/ocs/ocp.py:67: in exec_oc_cmd
out = run_cmd(cmd=oc_cmd)

cmd = ['oc', '-n', 'openshift-storage', '--kubeconfig', '/home/rperiyas/Documents/tools/OCS4x/auto_frame/cluster/auth/kubeconfig', 'create', ...]
kwargs = {}
r = CompletedProcess(args=['oc', '-n', 'openshift-storage', '--kubeconfig', '/home/rperiyas/Documents/tools/OCS4x/auto_fra...oes not match the namespace "openshift-storage". You must pass '--namespace=rook-ceph' to perform this operation.\n')

def run_cmd(cmd, **kwargs):
    """
    Run an arbitrary command locally

    Args:
        cmd (str): command to run

    Raises:
        CommandFailed: In case the command execution fails

    Returns:
        (str) Decoded stdout of command

    """
    log.info(f"Executing command: {cmd}")
    if isinstance(cmd, str):
        cmd = shlex.split(cmd)
    r = subprocess.run(
        cmd,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        stdin=subprocess.PIPE,
        **kwargs
    )
    log.debug(f"CMD output: {r.stdout.decode()}")
    if r.stderr:
        log.error(f"CMD error:: {r.stderr.decode()}")
    if r.returncode:
        raise CommandFailed(

          f"Error during execution of command: {cmd}."

            f"\nError is {r.stderr.decode()}"
        )

E ocs.exceptions.CommandFailed: Error during execution of command: ['oc', '-n', 'openshift-storage', '--kubeconfig', '/home/rperiyas/Documents/tools/OCS4x/auto_frame/cluster/auth/kubeconfig', 'create', '-f', 'templates/ocs-deployment/temp.yaml', '-o', 'yaml'].
E Error is error: the namespace from the provided object "rook-ceph" does not match the namespace "openshift-storage". You must pass '--namespace=rook-ceph' to perform this operation.

../../ocs-ci/utility/utils.py:655: CommandFailed`

Version: Form May30th ocs-ci master branch

Existing Automation uses the rook, ceph and OCP images hard coded.
There should be a config file which should take parameter from user to deploy user required rook, ceph and OCP images/builds.

It would be worth to print particular version of downloaded openshift-installer and openshift-client into log.

ocs-ci/docs/getting_started.md: "python3.7 -m .venv <path/to/venv>" does not work. It fails with error message "/usr/local/bin/python3.7: Relative module names not supported".

Fix: remove "." in ".venv" in above command.

Correct command: "python3.7 -m venv <path/to/venv>"

Currently we have two functions that do waiting for a certain condition: wait_for_resource_status and wait_for_func_status. Wouldn't it be nicer to use https://pypi.org/project/wait_for/ instead?

With the merging of PR #58 , OCS deployment is CSI based which creates Ceph pods(MONs, OSD, MGR), CSI pods, rook pods and toolbox pods.

But, In a setup deployed using this method, only RBD volumes can be created. To have Cephfs volumes, we need MDS pods to be part of deployment.

As the requirement docs lists both RBD and CephFS and default volume types, it would be helpful to have a basic set of MDS pods running in the cluster from day-0 ( as part of OCS deployment).

Things needed:

1. filesystem.yaml
2. oc create -f filesystem.yaml

Pytest run fails on Defining 'pytest_plugins' in a non-top-level conftest is no longer supported:

(venv) [ocsqe@localhost ocs-ci]$ python -m pytest -m deployment --cluster-name mbukatov-ocsqe                                                                 
==================================================================== test session starts =====================================================================
platform linux -- Python 3.7.3, pytest-4.5.0, py-1.8.0, pluggy-0.11.0 -- /home/ocsqe/projects/ocs-ci/venv/bin/python                                          
cachedir: .pytest_cache                                                                                                                                       
rootdir: /home/ocsqe/projects/ocs-ci, inifile: pytest.ini                                                                                                     
plugins: ocs-ci-0.1                                                                                                                                           
collected 0 items / 1 errors                                                                                                                                  
                                                                                                                                                              
=========================================================================== ERRORS ===========================================================================
_______________________________________________________________ ERROR collecting test session ________________________________________________________________
Defining 'pytest_plugins' in a non-top-level conftest is no longer supported because it affects the entire directory tree in a non-explicit way.              
  /home/ocsqe/projects/ocs-ci/unittests/conftest.py                                                                                                           
Please move it to a top level conftest file at the rootdir:                                                                                                   
  /home/ocsqe/projects/ocs-ci                                                                                                                                 
For more information, visit:                                                                                                                                  
  https://docs.pytest.org/en/latest/deprecations.html#pytest-plugins-in-non-top-level-conftest-files                                                          
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================== 1 error in 0.14 seconds ===================================================================
(venv) [ocsqe@localhost ocs-ci]$

This is happening because main pytest run is collecting tests from unittests directory.

Description of the problem

Utility function which downloads and installs openshift installer also overrides README file of ocs-ci repository.

Reproducer

1. clone ocs-ci repo
2. prepare all configuration (such as pull secret, aws config, see docs for full details)
3. run python -m pytest -m deployment --cluster-name example-ocs-deployment

Actual Results

README file in root directory of ocs-ci contains README file of openshift installer instead of one file for ocs-ci:

[ocsqe@localhost ocs-ci]$ git status -sb
## local
 M README.md
[ocsqe@localhost ocs-ci]$ git diff
diff --git a/README.md b/README.md
index e3aa2c3..a2ca940 100644
--- a/README.md
+++ b/README.md
@@ -1,9 +1,18 @@
-# OCS-CI
+# OpenShift Install
 
-OCS CI is a framework to test Red Hat OCS features and functionality using AWS
-and other supported platforms. The framework is based on CephCI
-  ( https://github.com/red-hat-storage/cephci )
+The OpenShift installer `openshift-install` makes it easy to get a cluster
+running on the public cloud or your local infrastructure.
 
-Documentation is structured and available under docs directory
-inside this project. You can click [HERE](docs/README.md) to see
-the Table Of Content(TOC) of our documentation.
\ No newline at end of file
+To learn more about installing OpenShift, visit [docs.openshift.com](https://docs.openshift.com)
+and select the version of OpenShift you are using.
+
+## Installing the tools
+
+After extracting this archive, you can move the `openshift-install` binary
+to a location on your PATH such as `/usr/local/bin`, or keep it in a temporary
+directory and reference it via `./openshift-install`.
+
+## License
+
+OpenShift is licensed under the Apache Public License 2.0. The source code for this
+program is [located on github](https://github.com/openshift/installer).
[ocsqe@localhost ocs-ci]$

Relevant logs:

14:09:51 - MainThread - utility.utils - INFO - Downloading openshift installer
14:10:30 - MainThread - utility.utils - INFO - Executing command: tar xzvf openshift-install.tar.gz  

Expected results

README file (and any other files tracked in ocs-ci repository) are not changed during the process.

It would be more reasonable to extract only the files we expect to use.

Currently, to override cluster_namespace for rook we need to specify cluster_namespace for each file it will good if we make cluster_namespace as generic so all rook pods will come under that namespace

Currently, I am using this conf:

https://paste.fedoraproject.org/paste/B-XgA0b0sfDgN3OCEuWMiw

Move all constants from defaults.py to there

probably check in the runner and bail out sooner...

2019-04-23 10:51:07,465 - test_ocs_basic_install - INFO - Executing command: oc cluster-info
2019-04-23 10:51:07,480 - __main__ - ERROR - Traceback (most recent call last):
  File "run.py", line 198, in run
    rc = test_mod.run(**test_kwargs)
  File "/Users/vasukulkarni/ocs-ci/tests/test_ocs_basic_install.py", line 80, in run
    run_cmd("oc cluster-info")
  File "/Users/vasukulkarni/ocs-ci/tests/test_ocs_basic_install.py", line 111, in run_cmd
    r = subprocess.run(cmd.split(), stdout=sys.stdout, stderr=sys.stderr, **kwargs)
  File "/Users/vasukulkarni/ocs-ci/venv2/lib/python3.7/site-packages/gevent/subprocess.py", line 1672, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Users/vasukulkarni/ocs-ci/venv2/lib/python3.7/site-packages/gevent/subprocess.py", line 627, in __init__
    restore_signals, start_new_session)
  File "/Users/vasukulkarni/ocs-ci/venv2/lib/python3.7/site-packages/gevent/subprocess.py", line 1505, in _execute_child
    raise child_exception
FileNotFoundError: [Errno 2] No such file or directory: 'oc'

Handle fresh install case and set the dir probably in ocp_install ?

INFO:test_ocs_basic_install:HEALTH_OK, install successful.
2019-05-16 13:26:01,785 - test_ocs_basic_install - INFO - HEALTH_OK, install successful.
DEBUG:ocsci.framework:Test case info:
{'abort_on_fail': True,
 'desc': 'Minimal installation of OCS',
 'destroy_cluster': False,
 'duration': 2624.847946166992,
 'end': 1558038361.7858372,
 'file': 'test_ocs_basic_install.py',
 'log_link': 'http://magna002.ceph.redhat.com/cephci-jenkins/cephci-run-1558035736932/install_OCS_0.log',
 'name': 'install OCS',
 'polarion_id': None,
 'post_results': False,
 'rp_service': None,
 'start': 1558035736.937891,
 'status': <TestStatus.PASSED: 0>,
 'suite_name': 'ocs_basic_install',
 'test_kwargs': {'config': {}, 'test_data': {'no-destroy': False}},
 'test_mod': <module 'test_ocs_basic_install' from '/Users/vasukulkarni/ocs/ocs-ci/tests/test_ocs_basic_install.py'>,
 'unique_name': 'install_OCS_0'}
INFO:utility.utils:Test logfile: /tmp/cephci-run-1558035736932/destroy_OCS_cluster_0.log
2019-05-16 13:26:01,787 - utility.utils - INFO - Test logfile: /tmp/cephci-run-1558035736932/destroy_OCS_cluster_0.log
INFO:utility.utils:Completed log configuration
2019-05-16 13:26:01,787 - utility.utils - INFO - Completed log configuration
INFO:ocsci.framework:Executing test case: destroy_OCS_cluster_0
2019-05-16 13:26:01,790 - ocsci.framework - INFO - Executing test case: destroy_OCS_cluster_0
INFO:test_destroy_cluster:Running OCS cluster destroy
2019-05-16 13:26:01,790 - test_destroy_cluster - INFO - Running OCS cluster destroy
ERROR:utility.utils:Traceback (most recent call last):
  File "/Users/vasukulkarni/ocs/ocs-ci/utility/utils.py", line 690, in destroy_cluster
    cluster_path = os.path.normpath(cluster_path)
  File "/Users/vasukulkarni/anaconda3/lib/python3.7/posixpath.py", line 333, in normpath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

2019-05-16 13:26:01,793 - utility.utils - ERROR - Traceback (most recent call last):
  File "/Users/vasukulkarni/ocs/ocs-ci/utility/utils.py", line 690, in destroy_cluster
    cluster_path = os.path.normpath(cluster_path)
  File "/Users/vasukulkarni/anaconda3/lib/python3.7/posixpath.py", line 333, in normpath
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType

File tests/helpers.py should be moved to some already existing python module with such functions.

OCP.exec_oc_cmd("describe pod name") returns:

/home/ec2-user/venv/lib/python3.7/site-packages/kubernetes/config/kube_config.py:496: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
config_dict=yaml.load(f),
Traceback (most recent call last):
File "get_ip_addrs.py", line 53, in
print(get_ip_addrs())
File "get_ip_addrs.py", line 46, in get_ip_addrs
OCP.exec_oc_cmd(oc_cmd)
File "/home/ec2-user/ocs-ci/ocs/ocp.py", line 66, in exec_oc_cmd
return munchify(yaml.safe_load(out))
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/init.py", line 162, in safe_load
return load(stream, SafeLoader)
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/init.py", line 114, in load
return loader.get_single_data()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/constructor.py", line 41, in get_single_data
node = self.get_single_node()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/composer.py", line 127, in compose_mapping_node
while not self.check_event(MappingEndEvent):
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
if self.check_token(KeyToken):
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/scanner.py", line 116, in check_token
self.fetch_more_tokens()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/scanner.py", line 223, in fetch_more_tokens
return self.fetch_value()
File "/home/ec2-user/venv/lib/python3.7/site-packages/yaml/scanner.py", line 579, in fetch_value
self.get_mark())
yaml.scanner.ScannerError: mapping values are not allowed here
in "", line 10, column 37:
Annotations: openshift.io/scc: rook-ceph

==============================================================

When I tried something invalid, I got the following:

home/ec2-user/venv/lib/python3.7/site-packages/kubernetes/config/kube_config.py:496: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
config_dict=yaml.load(f),
CMD error:: Error: unknown command "xxxxxxx" for "oc"
Run 'oc --help' for usage.

Traceback (most recent call last):
File "get_ip_addrs.py", line 53, in
print(get_ip_addrs())
File "get_ip_addrs.py", line 46, in get_ip_addrs
OCP.exec_oc_cmd(oc_cmd)
File "/home/ec2-user/ocs-ci/ocs/ocp.py", line 65, in exec_oc_cmd
out = run_cmd(cmd=oc_cmd)
File "/home/ec2-user/ocs-ci/utility/utils.py", line 654, in run_cmd
f"Error during execution of command: {cmd}"
ocs.exceptions.CommandFailed: Error during execution of command: ['oc', '-n', 'openshift-storage', '--kubeconfig', '/tmp/ocs-ci-cluster-ec2-user-30031/auth/kubeconfig', 'xxxxxxx', 'describe', 'pod', 'rook-ceph-agent-5h67j']

==============================================================

So I tried running the valid corresponding command from the shell and it worked:

oc -n openshift-storage --kubeconfig /tmp/ocs-ci-cluster-ec2-user-30031/auth/kubeconfig describe pod rook-ceph-agent-5h67j
Name: rook-ceph-agent-5h67j
Namespace: openshift-storage
Priority: 0
PriorityClassName:
Node: ip-10-0-166-194.us-east-2.compute.internal/10.0.166.194
Start Time: Tue, 28 May 2019 22:44:03 +0000
Labels: app=rook-ceph-agent
controller-revision-hash=6848b56bb
pod-template-generation=1
Annotations: openshift.io/scc: rook-ceph
Status: Running
IP: 10.0.166.194
Controlled By: DaemonSet/rook-ceph-agent
Containers:
rook-ceph-agent:
Container ID: cri-o://ba7d0de2fd216359bd7cabc5ec2f21ef445defbca3ef90dcc6e72c4ec8388f30
Image: quay.io/rhceph-dev/rook
Image ID: quay.io/rhceph-dev/rook@sha256:ccad618bedd562354beef8a6452510e50bfa1da9c30e0360e4ce75f06a84a8d7
Port:
Host Port:
Args:
ceph
agent
State: Running
Started: Tue, 28 May 2019 22:45:05 +0000
Ready: True
Restart Count: 0
Environment:
POD_NAMESPACE: openshift-storage (v1:metadata.namespace)
NODE_NAME: (v1:spec.nodeName)
AGENT_MOUNT_SECURITY_MODE: Any
ROOK_ENABLE_SELINUX_RELABELING: true
ROOK_ENABLE_FSGROUP: true
Mounts:
/dev from dev (rw)
/flexmnt from flexvolume (rw)
/lib/modules from libmodules (rw)
/sys from sys (rw)
/var/run/secrets/kubernetes.io/serviceaccount from rook-ceph-system-token-w9kxj (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
flexvolume:
Type: HostPath (bare host directory volume)
Path: /etc/kubernetes/kubelet-plugins/volume/exec
HostPathType:
dev:
Type: HostPath (bare host directory volume)
Path: /dev
HostPathType:
sys:
Type: HostPath (bare host directory volume)
Path: /sys
HostPathType:
libmodules:
Type: HostPath (bare host directory volume)
Path: /lib/modules
HostPathType:
rook-ceph-system-token-w9kxj:
Type: Secret (a volume populated by a Secret)
SecretName: rook-ceph-system-token-w9kxj
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/network-unavailable:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unschedulable:NoSchedule
Events:

I am wondering if the fields at the end with colons and no spaces are confusing things.

Couldn't find --cred option in run.py:

(ocs-venv) [ebenahar@localhost ocs-ci]$ python3.7 run.py --suite suites/ocs_basic_install.yml --cred ~/.aws/config --log-level info
Usage:
run.py --suite FILE
[--conf ]
[--store]
[--reuse ]
[--post-results]
[--report-portal]
[--log-level ]
[--cluster-name ]
[--no-email]
run.py --cleanup=NAME [--osp-cred ]
[--log-level ]

For example, functions like pv create and delete, which are currently implemented in tests/test_pv_creation.py [1], should be in a commonplace.

[1] https://github.com/red-hat-storage/ocs-ci/pull/43/files/dbf2d5d41b69c9a032385cca66c549a3a8f3a032#diff-2b4e26e7e510a40da374e0d8ece59f04

We need the ability to integrate our testing framework with:

• Report Portal
• Polarion

to report our test results. In addition to this we would also like the ability to send email report summaries.

RHV has apparently successfully done this, we need to investigate what they have and utilize/re-purpose what they have to fit our needs.

currently, the number of worker nodes installed is 6. I don't think we need 6 worker nodes as the default value to run functional tests. So, Unless explicitly asked for default installation should have 3 worker nodes.

Currently, the OCS-CI deploys OCP along with rook+ceph , which is flexvolume based.
Requesting to add an option for deploying rook based ceph-csi on similar OCP 4.x environment.

The high-level steps for deployment will be: ( once an OCP cluster is UP)

cd /root/rook/cluster/examples/kubernetes/ceph
oc create -f common.yaml
oc apply -f ./csi/rbac/rbd/ ; oc apply -f ./csi/rbac/cephfs/
oc create -f operator-openshift-with-csi.yaml
oc create -f cluster.yaml
oc create -f toolbox.yaml
oc create -f filesystem.yaml
oc create -f object.yaml

cd ./csi/example/rbd
oc create -f pool.yaml (copy pool.yaml from cluster/examples/kubernetes/ceph
oc create -f secret.yaml
oc create -f storageclass.yaml #make sure to edit base64 secret and rbd pool name
oc create -f pvc.yaml
oc create -f pod.yaml

CephFS -sc and security key creation

cd ./csi/example/cephfs
oc create -f secret.yaml
oc create -f storageclass.yaml #make sure to edit base64 secret and fs data pool name
oc create -f pvc.yaml
oc create -f pod.yaml

Thanks,
Neha

We need to move our current implementation of --destroy to pytest way to allow us to run it via pytest.

Not sure how much time it took, but when I reran the condition was met.

2019-05-09 19:52:26,809 - test_ocs_basic_install - INFO - Waiting 5 seconds...
INFO:utility.utils:Executing command: oc wait --for condition=ready pod -l app=rook-ceph-operator -n openshift-storage --timeout=120s
2019-05-09 19:52:31,816 - utility.utils - INFO - Executing command: oc wait --for condition=ready pod -l app=rook-ceph-operator -n openshift-storage --timeout=120s
DEBUG:utility.utils:CMD output: 
ERROR:utility.utils:CMD error:: error: no matching resources found

2019-05-09 19:52:32,331 - utility.utils - ERROR - CMD error:: error: no matching resources found

ERROR:ocsci.framework:Traceback (most recent call last):
  File "/Users/vasukulkarni/ocs/ocs-ci/ocsci/framework.py", line 116, in execute
    rc = self.test_mod.run(**self.test_kwargs)
  File "/Users/vasukulkarni/ocs/ocs-ci/tests/test_ocs_basic_install.py", line 134, in run
    "oc wait --for condition=ready pod "
  File "/Users/vasukulkarni/ocs/ocs-ci/utility/utils.py", line 650, in run_cmd
    f"Error during execution of command: {cmd}"
ocs.exceptions.CommandFailed: Error during execution of command: ['oc', 'wait', '--for', 'condition=ready', 'pod', '-l', 'app=rook-ceph-operator', '-n', 'openshift-storage', '--timeout=120s']

2019-05-09 19:52:32,335 - ocsci.framework - ERROR - Traceback (most recent call last):
  File "/Users/vasukulkarni/ocs/ocs-ci/ocsci/framework.py", line 116, in execute
    rc = self.test_mod.run(**self.test_kwargs)
  File "/Users/vasukulkarni/ocs/ocs-ci/tests/test_ocs_basic_install.py", line 134, in run
    "oc wait --for condition=ready pod "
  File "/Users/vasukulkarni/ocs/ocs-ci/utility/utils.py", line 650, in run_cmd
    f"Error during execution of command: {cmd}"
ocs.exceptions.CommandFailed: Error during execution of command: ['oc', 'wait', '--for', 'condition=ready', 'pod', '-l', 'app=rook-ceph-operator', '-n', 'openshift-storage', '--timeout=120s']

I see few minor issues with configuration, which includes:

• There are 2 directories: conf and config, it would be nice to unify them into single directory or drop one if one is not needed.
• Directory osp contains 2 yaml files with some configuration, without much explanation. Could this be dropped or moved somewhere?
• File conf/ocs_basic_install.yml seems to have no effect, as it notes that 6 worker and 6 master nodes are created in aws, but when one runs the installation, only 3 of each are created. We may either drop the file to fix the setup so that values from there are followed. Resoved by #57 and 07b5eec
• Another config file is .ocs-ci.yaml created via cp ocs-ci.yaml.template ~/.ocs-ci.yaml as noted in docs/getting_started.md but I don't see problem with that.

I'm reporting this in a single issue, as the problems listed above are intertwined.

We have CLI arguments for specifying a cluster configuration, suite configuration, as well as the path to an existing cluster directory in the event you want to run tests against an established cluster. We need to create custom parser arguments so we may do the same through pytest, as well as implement the necessary functionality to support them in our tests.

This issue might be broken up into several based on the scope of things, but I wanted to capture the need for all three.

• cluster_config
• cluster_path
• suite_config

While writing test cases, We may have to run ceph commands to get to know something about the ceph cluster - health, size, pools, etc. So in our test framework, I expect a function I can call and this will internally go find the ceph toolbox pod's name, convert it into 'oc exec ' and give me the output in json format which can be parsed.

Ceph failure domain should be "host" in storage-manifest.yaml.

With OSD level failure domain, failure of one node with multiple OSDs will lead to IO failure and data unavailability.

Replica size should be 3.

As we complete our transition to utilizing pytest we will want to remove any code that was only applicable to our custom test runner. This should be done once test execution and all features we utilized with our custom runner are supported through pytest.

Items to remove (not limited to):

• TestStatus
• TestCase Class
• run.py
• several utility functions

There are cases where one might need to use ocp.get method to check if a resource is already available. The get method fails with CMD error if the resource is not available. If the method returns False, get method can be used to check if a resource is available before trying to create it again.

For eg., I want to create a new project and I'd like to check if the project already exists in the cluster.

This is how I'd like to check.

 	def create_namespace(**kwargs):
  	    pdb.set_trace()
  	    project_name = (kwargs['project_name'])
  	    **if NAMESPACE.get(project_name):**
  	        log.info(f'project {project_name} exists, using the existing namespace  ')
  	        return True

Right now we have every yaml in "templates/ocs-deployment", based on the classes we need to separate the deployment yaml's from other class based yaml's so that organization is clear.

With the WIP on the following two PRs, we would need to have standard jinja2 templates in place under templates folder.

1. #58
2. #95

Hence, this issue is for the need to submit the 4 jinja2 templates :

1. csi-rbd-storageclass.yaml
2. csi-rbd-secret.yaml
3. csi-cephfs-storageclass.yaml
4. csi-cephfs-secret.yaml

Since request https://github.com/red-hat-storage/ocs-ci/pull/142/files the docs notes that it should be possible to run the pytest wrapper via comman line tool run-ci (which is created via setuptools during pip based installation in a virtual env).

But this doesn't work, because the module run_ocsci.py is not installed anywhere (via setuptools), so the run-ci wrapper wont work:

(venv) [ocsqe@localhost ocs-ci]$ run-ci -h
Traceback (most recent call last):
  File "/home/ocsqe/projects/ocs-ci/venv/bin/run-ci", line 11, in <module>
    load_entry_point('ocs-ci==0.1', 'console_scripts', 'run-ci')()
  File "/home/ocsqe/projects/ocs-ci/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/home/ocsqe/projects/ocs-ci/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 2793, in load_entry_point
    return ep.load()
  File "/home/ocsqe/projects/ocs-ci/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 2411, in load
    return self.resolve()
  File "/home/ocsqe/projects/ocs-ci/venv/lib64/python3.7/site-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'run_ocsci'

While the explicit run of the module works as expected:

$ python -m run_ocsci -m deployment --cluster-name mbukatov-ocs-deployment tests

Resolution:

• revert https://github.com/red-hat-storage/ocs-ci/pull/142/files
• make the wrapper executable directly without setuptools (simple and explicit) or make it installed via setuptools

Doc link - https://github.com/red-hat-storage/ocs-ci/blob/master/docs/getting_started.md

The document doesn't mention to clone the OCS-CI repo before asking to install " pip install -r requirement.txt". It would be helpful to have this information for first time users.

Doc link

Getting Started ->Installing -> Step 2 : It mentions running "pip install -r requirements.txt" .

Current content:

1. Setup a python 3.7 virtual environment. This is actually quite easy to do now.
   --snip--

2. Install requirements with pip install -r requirements.txt

Change suggested:

Please add a comment to clone the repo before trying to execute Step#2.

1. Clone the ocs-ci repository and run "pip install -r requirements.txt" ( or any other equivalent statement for user's clear understanding)

Thanks,
Neha

We should implement the hierarchy design we will agree on.
This was raised by @RazTamir along with few design options:
https://docs.google.com/drawings/d/1Esc0zexs9Oe8BaKQA71LuV5-K1dAJSSluWq_aCieGrM/edit
https://docs.google.com/drawings/d/1SUbmE-TITkgWNruiKcFwrl52ULEuO_rDUtY4guIz2UY/edit

In templates/ocs-deployment/storage-manifest.yaml, we specify storage class as:

  clusternamespace: {{ cluster_namespace | default('openshift-storage') }}

This should read clusterNamespace instead. Otherwise the namespace selected is not
recognized and a default namespace is used instead by rook.

We can use pytest ordering plugin to make sure that deployment test is running as the first one test, and the destroy cluster will be run at the end of the execution. We can specify execution for example like: pytest -m "deployment or (tier1 and manage) or destroy" to run deployment, then tier1 tests of manage team and destroy the cluster.

We need to integrate Bugzilla to our framework so test cases will be skipped in case there's an open bug.

The implementation in RHV already exists so we can use it

We should follow:
https://docs.openshift.com/container-platform/4.1/web-console/working-with-projects.html#creating-a-project-using-the-CLI_projects

Just noticed that we have to set jenkins_rc to proper enum, right now it will just return 0 regardless of test pass or fail.

Currently with our ocs-ci custom test runner we create a log FileHandler for every test case. We would like to replicate this behavior with pytest. One way of getting after this that sounds fairly straight forward is implementing a function level fixture that creates and removes the handlers for every test.

https://stackoverflow.com/questions/41400722/pytest-implementing-a-logfile-per-test-method

Description of the problem

Extra volumes are not deleted during cluster destroy.

Steps to reproduce

1. Create cluster, but disable cluster destroy: python run.py --suite suites/ocs_basic_install.yml --log-level info --cluster-name=loginname-ocs4-testing-cluster --no-email --no-destroy
2. Destroy cluster via: python run.py --log-level DEBUG --cluster-path /tmp/loginname-ocs4-testing-cluster-88528/ --destroy

Actual results

3 extra volumes created in the step 1 still exist:

mbukatov-ocs4-testing-fmc7p-worker-us-east-2a-9hr9r_extra_volume
mbukatov-ocs4-testing-fmc7p-worker-us-east-2b-rgfx5_extra_volume
mbukatov-ocs4-testing-fmc7p-worker-us-east-2c-97vjn_extra_volume

Expected results

There are no extra volumes left.

Most of our test cases will have a step to attach one or more PVC to application pod. It is a good idea to implement this functionality as a common function.

Make sure we have a README file with short explanation and references for every directory in the root of the repository:

• conf: already has one via https://github.com/red-hat-storage/ocs-ci/pull/110/files
• docs: already there
• mita: something along the lines of #100 (comment) (credit and licence should be added as a part of this)
• oc
• python modules: ocs, ocsci
• pytest_customization: #171
• suites
• templates
• tests
• unittests: #163
• utility

The README file itself could be quite short, with one or two sentences and at least one link to documentation of related libraries or tools.

During deployment, log version of ocs/rook images. It would be useful to include hash along with tag name for each image. Mentioning pull url would be also useful.

References:

• https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/

We have mita directory, likely a configuration of https://github.com/ceph/mita, which we no longer use. It should be safe to drop it.