Verify test coverage about qiskit-ibm-runtime HOT 4 CLOSED

#110 will enable code coverage reports for both, unit and integration tests. Let's continue to use #22 as a follow-up to make it fancier and easier to spot trends over time (e.g. by adding coveralls integration, ...). In a first step we need to manually download a workflow artifact to review coverage metrics. This is not ideal but probably a first step towards betters insights.

What remains to be done to meet the acceptance criteria here

• review the unit and integration test coverage reports (see #110 (comment) for details)
• decide if the current coverage is "sufficient"
• if necessary: identify improvement areas as needed and prioritize follow-up tasks

from qiskit-ibm-runtime.

Let's pick up work here again as soon as the referenced PR (#110) is integrated and the push tests do no longer consistently fail (tracked via #93).

from qiskit-ibm-runtime.

Gathered test coverage information from one of the rare successful integration test runs to unblock working on this issue:
Unit test coverage report - python3.9-ubuntu-latest.zip
Integration test coverage report - legacy-production.zip
Integration test coverage report - cloud-production.zip

Downloaded from a CI run of #114

from qiskit-ibm-runtime.

Data

Coverage Report - Unit Tests

Coverage Report - Integration tests run against IBM Quantum

Coverage Report - Integration tests run against IBM Cloud

Assessment

• Overall test coverage
  • 57% for unit tests: 57%
  • 62% for integration tests against IBM Cloud
  • 66% for integration tests against IBM Quantum
• The public API entrpoints in (ibm_runtime_service) are well-covered (80-89%)
• Modules with little to now test coverage have not yet been enabled yet (visualization / jupyter), should improve via #111 and #112

Though we can always improve, I personally think we're good for a first release and these reports along with the planned enhancements will make sure we keep an eye on coverage metrics anyway.

@jyu00, @rathishcholarajan anything else you see as an open to do here? Otherwise, can we close it?

from qiskit-ibm-runtime.