Exiting Python session generates TypeError about qiskit-ibm-runtime HOT 3 CLOSED

That's a tricky one. Nailed it down as below and will continue the investigation early next week.

good-case.py

from requests import Session


class CustomSession(Session):
    def __del__(self) -> None:
        self.close()


session = CustomSession()
result = session.get("https://auth.quantum-computing.ibm.com/api/version").json()
print(result)

bad-case.py

from qiskit_ibm_runtime.api.session import RetrySession


session = RetrySession(
    base_url="https://auth.quantum-computing.ibm.com",
    auth=None,
)
# TypeError only seen when a request is made
result = session.get("/api/version").json()
print(result)

Running python good-case.py succeeds, while python bad-case.py fails with the reported TypeError. I.e. something within RetrySession must be the culprit...

from qiskit-ibm-runtime.

Investigation Results

Observations

• Code fragment that is causing TypeError: catching classes that do not inherit from BaseException. In the failing case, queue.Empty is None.

# File ".../site-packages/urllib3/connectionpool.py", line 494, in close
def close(self):
   # ...
-> except queue.Empty:
      pass

• Entrypoint from code within qiskit-ibm-runtime that leads to the exception:

# File ".../qiskit_ibm_runtime/api/session.py", line 173, in __del__
class RetrySession(Session):
    def __del__(self) -> None:
->        self.close()

• To debug this further I instrumented the custom session code with a print statement as shown below.

import queue
class CustomSession(Session):
    def __del__(self) -> None:
        print("CustomSession#__del__", queue.Empty)
        self.close()

# output in success case: CustomSession#__del__ <class '_queue.Empty'>
# output in failing case: RetrySession#__del__ None

Conclusions

Our code calls session cleanup code from within the destructor of a custom session subclass. The destructor (__del__) is called when the object is destroyed. At this point in time it can apparently happen that queue.Empty returns None. Maybe because Python has cleaned up other objects as well.

I can't find any references which indicate that triggering the cleanup of session information from within the destructor is a supported best practice. At the same time we can't switch to using a session context manager without breaking the APIs that consumers are used to.

Proposed Solution

Ignore errors and do not spend additional time understanding how python cleans up module references under the covers (see #103)

from qiskit-ibm-runtime.

With #103 the "high priority" part of this issue has been addressed.

Based on the above investigation we can reprioritize the work and come up with a proper fix in a next step. Need a second pair of eyes to help with that and think about clean solutions (cc: @rathishcholarajan)

from qiskit-ibm-runtime.