Standard deviation parameters are incorrectly treated as variances in statespace covariance matrices about pymc-experimental HOT 24 CLOSED

from pymc-experimental.

CC @jessegrabowski

from pymc-experimental.

I can take a look, but this is just a debug function. Why are you using it? There is a large test suite that checks that the statespace models match the statsmodels outputs.

from pymc-experimental.

I use it because it makes data generation for a given state-space model easy, much better than statsmodel (as far as I know).

from pymc-experimental.

In this specific case, statsmodels expects sigma squared, whereas the structural model expects sigma. If you remove the square root from the structural model (but not the statsmodel argument), you will get the same output, modulo random draws. I don't think setting the seed is enough to guarantee the same outputs between the two functions, because there might be differences in how/when each random number is generated (all at once vs one-by-one in loop, for example).

I'm not sure what your exact use-case in, but in general I suggest you use the usual PyMC API to generate draws from the statespace models, rather than the testing function.

from pymc-experimental.

from pymc-experimental.

measurement_error = st.MeasurementError(name="obs")
IRW = st.LevelTrendComponent(order=2, innovations_order=[0, 1])

# Change sigma_trend to remove the square root
param_dict = {
"initial_trend": [0., 0.01], "sigma_trend": np.array([5e-7]),
"sigma_obs": np.array([0.]),
}

mod = IRW + measurement_error
nobs = 1024 # we simulate a time series of length Nobs

x, y = simulate_from_numpy_model(mod, np.random.default_rng(2163), param_dict, steps=nobs)

# sm part unchanged 

fig, ax = plt.subplots()
ax.plot(y, label='Structural')
ax.plot(y_sim, label='Statsmodels')
ax.legend()
plt.show()

This now looks correct to me?

from pymc-experimental.

from pymc-experimental.

Yeah that's a fair point, the name is not consistent with what you get. Currently, whatever number you pass in is directly plugged into the Q matrix at whatever position on the main diagonal, so either it should be squared internally (this is what statsmodels does) or the name should be changed.

My preference would be to square things internally and keep the name the same, because no other PyMC distributions are parameterized by the variance directly. Does that seem reasonable?

from pymc-experimental.

from pymc-experimental.

from pymc-experimental.

It applies to all shocks in all structural models I believe. I will have to check them all.

from pymc-experimental.

from pymc-experimental.

You'll have to pip install the main branch from github. We haven't done a new release yet, so just doing pip install pymc-experimental won't have the patch

from pymc-experimental.

from pymc-experimental.

It's similar to what you did before when you were installing from my branch, but now just targeting the main branch. Should be something like:

pip install git+https://github.com/pymc-devs/pymc-experimental.git

from pymc-experimental.

from pymc-experimental.

Make sure you pip uninstall pymc-experimental first

from pymc-experimental.

from pymc-experimental.

from pymc-experimental.

I just tested the command on my computer and it seems like it works. There hasn't been a release, so the version won't be bumped by this command.

You can check the source code by e.g. st.MeasurementError?? (in a Jupyter notebook) and check that you see the sigma being squared in the make_symbolic_graph method, like this:

    def make_symbolic_graph(self) -> None:
        error_sigma = self.make_and_register_variable(f"sigma_{self.name}", shape=(self.k_endog,))
        diag_idx = np.diag_indices(self.k_endog)
        idx = np.s_["obs_cov", diag_idx[0], diag_idx[1]]
        self.ssm[idx] = error_sigma**2

from pymc-experimental.

from pymc-experimental.

Everything works as it appears now. You put a prior on sigma (the standard deviation), which is then squared and placed on the diagonal of the covariance matrix.

from pymc-experimental.

from pymc-experimental.