Calculation of ys_lower and ys_upper about qiskit-experiments HOT 9 CLOSED

I'll add a monotonicity parameter to plot_curve_fit

from qiskit-experiments.

@nkanazawa1989 emphasizes the importance of confidence interval also for non-monotone functions. Itoko-san (ccing him here is not working) has proposed a way to do it:

ys_error**2 = diagonal elements of [residual_chisq * W**-1 - J * pcov * J.T]

where J is Jacobian and pcov is covariance matrix (this is provided by analysis option). This is equation 208 in the paper:
CI_NL_WLS_imm2804.pdf

Current status is that the existing implementation is correct for T1 and RB, while all the rest are non-monotone. Therefore we are thinking of the following plan:

1. Close #149.
2. Open a new, hopefully quick and temporary PR, where experiments other than T1 and RB will raise a warning when drawing the confidence interval (@nkanazawa1989 prefers this option over another option, which is to draw the confidence interval only for T1 and RB).
3. Implement Itoko-san's idea.

@nkanazawa1989 thinks it's important to do it for the first release (including no. 3), I'm not sure about it.

from qiskit-experiments.

Thanks @yaelbh for handling this issue.

I still prefer to have error interval (usually uncertainty propagation differs from stdev of each data point) and this is convenient to grab the goodness of experiment setting we provide, for example, when we set short Clifford lengths for RB, the fit image will show wider error region though each stdev of data point is small. This can be also understood by checking fit value's stdev, but visualization is especially important for result-db.

If implementation takes long time (needs some time to develop a concrete theoretical background), removing confidence interval for the first release is reasonable decision. In this case, the step 2 will just override plot_cirve_fit with confidence_interval: bool = False.

from qiskit-experiments.

I'm not quite sure that the above equation I proposed is correct. Please double- or triple-check. I personally think we should drop confidence_interval from the option of plot_curve_fit until we find the right name and definition of "error region around estimated ys" in plotting.

from qiskit-experiments.

+1 to dropping confidence_interval for the first release. Anyways, is confidence_interval the right name? To me, it sounds like a technical term used in statistics, but at least the current definition seems different from the confidence interval in statistics...

from qiskit-experiments.

The term "confidence interval" appears also in a comment in curve_analysis.py, I think with a different meaning from both statistics and the plot_curve_fit parameter:

            # automatic scaling y axis by actual data point.
            # note that y axis will be scaled by confidence interval by default.

from qiskit-experiments.

We should consider adding yet a new set of options: plot_options

from qiskit-experiments.

An initial fix was done in #163. Therefore, I'm removing the "Release 0.1" milestone from this issue.

from qiskit-experiments.

@itoko @nkanazawa1989 I'm moving this issue to the IceBox pipeline Please move it back to Backlog if you think it will be addressed in the near future.

from qiskit-experiments.