If I recall correctly you found all the material online after all. If so, we can set the repo to private without checking with John first.

We need to incorporate Newton–Kantorovich (NK) iterations in our value function iteration process. @MaxBlesch, please review all elements of our code in light of the replication material provided here
https://www.econometricsociety.org/publications/econometrica/2016/01/01/comment-%E2%80%9Cconstrained-optimization-approaches-estimation

With #42 and #43 a new interface and new features come in place. We need to adjust the documentation accordingly and setup promotion material for the comparison of MPED and NFXP.

We want to separate model and estimation code, to make ruspy more adaptive to different estimation packages. This means we want to eliminate all calls of optimize functions like estimagic.minimize for the NFXP and ipopt and nlopt for the MPEC in the estimation code.

ToDo:

• The estimate function has to be rewritten to a get_criterion_function function, which returns for NFXP and MPEC the corresponding criterion function and its derivative together with their arguments besides params. (Maybe even just return the criterion and derivative, as partial functions with params as single input).
• Rewrite interface, such that the init_dict only contains model specifications and fixed point algorithm details (alg_details in code).
• Rewrite code, such that params is a numpy array.
• Rewrite tests, such that they only test the criterion function. The optimization of it will not be tested, as it is not part of the package any more. However, the correct estimation is somehow tested by more advanced tutorials.
• Adapt documentation and tutorials.

To bolster our narrative about OSE Education it would be useful for all packages to have a page that provides information about the master thesis that contributed to it.

=============================================================================================================================== test session starts ===============================================================================================================================
platform linux -- Python 3.6.8, pytest-4.4.0, py-1.7.0, pluggy-0.9.0
rootdir: /home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy
plugins: xdist-1.26.1, forked-0.2
collected 0 items / 1 errors

===================================================================================================================================== ERRORS ======================================================================================================================================
__________________________________________________________________________________________________________________________ ERROR collecting test session __________________________________________________________________________________________________________________________
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/config/init.py:435: in _importconftest
return self._conftestpath2mod[conftestpath]
E KeyError: local('/home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy/ruspy/test/conftest.py')

During handling of the above exception, another exception occurred:
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/config/init.py:441: in _importconftest
mod = conftestpath.pyimport()
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/py/_path/local.py:668: in pyimport
import(modname)
ruspy/init.py:6: in
from ruspy.simulation.simulation import simulate
ruspy/simulation/simulation.py:8: in
import mpmath as mp
E ModuleNotFoundError: No module named 'mpmath'

During handling of the above exception, another exception occurred:
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/py/_path/common.py:377: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/py/_path/common.py:429: in gen
for p in self.gen(subdir):
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/py/_path/common.py:418: in gen
dirs = self.optsort([p for p in entries
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/py/_path/common.py:419: in
if p.check(dir=1) and (rec is None or rec(p))])
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/main.py:660: in _recurse
ihook = self.gethookproxy(dirpath)
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/main.py:481: in gethookproxy
my_conftestmodules = pm._getconftestmodules(fspath)
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/config/init.py:419: in _getconftestmodules
mod = self._importconftest(conftestpath.realpath())
/home/peisenha/.local/share/anaconda3/envs/ruspy/lib/python3.6/site-packages/_pytest/config/init.py:458: in _importconftest
raise ConftestImportFailure(conftestpath, sys.exc_info())
E _pytest.config.ConftestImportFailure: (local('/home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy/ruspy/test/conftest.py'), (<class 'ModuleNotFoundError'>, ModuleNotFoundError("No module named 'mpmath'",), <traceback object at 0x7fd109710708>))
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

peisenha@pontos:/.../pkl/replication_data$ ls -al
total 396
drwxr-xr-x 2 peisenha peisenha 4096 Dez 27 15:52 .
drwxr-xr-x 4 peisenha peisenha 4096 Dez 27 15:52 ..
-rw-r--r-- 1 peisenha peisenha 9847 Dez 27 15:52 'RepGroup 1.pkl'
-rw-r--r-- 1 peisenha peisenha 5548 Dez 27 15:52 'RepGroup 2.pkl'
-rw-r--r-- 1 peisenha peisenha 81496 Dez 27 15:52 'RepGroup 3.pkl'
-rw-r--r-- 1 peisenha peisenha 104752 Dez 27 15:52 'RepGroup 4.pkl'
-rw-r--r-- 1 peisenha peisenha 37135 Dez 27 15:52 'RepGroup 5.pkl'
-rw-r--r-- 1 peisenha peisenha 31087 Dez 27 15:52 'RepGroup 6.pkl'
-rw-r--r-- 1 peisenha peisenha 55279 Dez 27 15:52 'RepGroup 7.pkl'
-rw-r--r-- 1 peisenha peisenha 55279 Dez 27 15:52 'RepGroup 8.pkl'
peisenha@pontos:/.../pkl/replication_data$

This special treatment, which appears to be required for an exact replication of Rust (1987), appears ad hoc.

We want to confirm this in the replication material for Rust (2016) and ask Fedor for advice during his visit.

conda env update -f environment.yml
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you.

We want to explore the ideas outlined in Jorgensen (2019).

We seem to implement the smooth ambiguity model as outlined in Marinacci (2015) and estimate its parameters.

We seek to implement the MPEC estimation as outlined in Sue & Judd (2012).

Analytical derivatives are available, please incorporate to speed up and increase precision of outer loop.

please fix

https://github.com/OpenSourceEconomics/ruspy/blob/master/ruspy/simulation/value_zero.py These are relvant for illustrative purposes only and thus will be moved outside the core package. The documentation``or promotion` folder. If they are needed for testing, we can also move them into https://github.com/OpenSourceEconomics/ruspy/tree/master/ruspy/test/resources

py.test
======================================================================== test session starts ========================================================================
platform linux -- Python 3.6.8, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
rootdir: /home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy
plugins: xdist-1.28.0, forked-1.0.2
collected 13 items

ruspy/test/regression_test.py . [ 7%]
ruspy/test/test_estimation.py .... [ 38%]
ruspy/test/test_replication.py ..... [ 76%]
ruspy/test/test_simulation.py ... [100%]

========================================================================= warnings summary ==========================================================================
ruspy/test/test_replication.py:22
/home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy/ruspy/test/test_replication.py:22: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
init_dict = yaml.load(y)

ruspy/test/test_simulation.py:21
/home/peisenha/external_storage/ownCloud/office/workspace/OpenSourceEconomics/ruspy/ruspy/test/test_simulation.py:21: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
init_dict = yaml.load(y)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============================================================== 13 passed, 2 warnings in 148.80 seconds ===

Make ruspy conda installable. Align README to respy and move ruspy to anaconda ose.

Transfer the looser convergence criterion to calc_fixp.

Delete rule of thumb for the definition of the state size and let the user explicit state the size in the init file.

I want us to split simulate explicitly in get_decision and simulation of experience given the decision rule ... this seems useful for us now as in our case the probabilities determining both are not the same

The code uses cost for values which are not simply the cost and isntead the comple value for the alternative.

Please be more precise in the notebook which tables you are reproducing. For example, you do not reproduce the first table in Rust's paper, instead Table IIa

Please set up our usual set of labels for issues, see soepy for an example.

We want travis to execute the notebooks in the promotion directory.

Add bootstrapping functionality again using estimagic as soon as they merged their bootstrap PR.

In the original algorithm, the algorithmic parameters for switching between NK and contractions iterations were changed after each run. This is a next step for ruspy and also the possibility of switching back and forth from the different iterations.

Once BHHH is in estimagic, we need to adjust promotional material, to promote full replication of Rust (1987) as well as Su and Judd's MPEC performance comparision.

Please integrate function call into the notebook. Also, the directories for the pkl files are not automatically created if they do not exist, thus the script cannot be run out of the box. Please fix.

Descriptives

The first two tables of the paper will be paritally replicated. To execute this file, please firts run the data_reading.py file in here.

Can we please consider removing this case distinction from the code base, it makes the code/interfaces more complicated without much gain. Can't we just go with Rust's convention and add a TODO: that notes your concerns/thoughts and then when there is some progress on that end we can change it afterwards?