RDR MultiCluster Job Deployment Failure - jsonschema.exceptions.ValidationError: 'up' was expected about ocs-ci HOT 4 CLOSED

Not enough logs provided. We need to enable COLLECT_LOGS option in the job I guess in order to collect must gather or keep cluster running.

2024-04-05 09:32:51  E           jsonschema.exceptions.ValidationError: 'up' was expected
2024-04-05 09:32:51  E           
2024-04-05 09:32:51  E           Failed validating 'const' in schema['properties']['status']:
2024-04-05 09:32:51  E               {'const': 'up'}
2024-04-05 09:32:51  E           
2024-04-05 09:32:51  E           On instance['status']:
2024-04-05 09:32:51  E               'down'

Clearly says that there was expected status up but it was down .

So if this is the case it's not the issue in the ocs-ci but in the deployment that i'ts not in expected state we want to to have it and validate for - that's why we have such check.

from ocs-ci.

So please provide a cluster and ping ecosystem team to have a look or attache MG Logs from cluster to confirm the above. @sidhant-agrawal

from ocs-ci.

@petr-balogh Another job failed with same error
COLLECT_LOGS and PAUSE_BEFORE_TEARDOWN options were enabled this time. We have the live cluster.
Job link: https://url.corp.redhat.com/3df3ed5

from ocs-ci.

Did debug the issue and we found out that OSD pod got up just 5 seconds based on the logs of pod before we did this schema validations, and OSD was still considered as DOWN status.

When I connected to provided cluster I see all OSDs are with status UP:
https://url.corp.redhat.com/0e37129

from ocs-ci.