Dear Team, please use pathlib for all our interactions with the filesystem. This ensures cross-platform comparability. Here is an example from respy., OpenSourceEconomics/respy@d7445d3

... supporting reading https://pandas-dev.github.io/pandas-blog/pandas-extension-arrays.html

I am not too happy with your use of the attr_dict here (and my use in respy). Instead, I would like us to try the use of a NamedTuple https://pymotw.com/2/collections/namedtuple.html for the immutable model_parameters. Let us discuss on our talk tomorrow ...

• access attributes directly and not by first creating local variables
• also interesting for the optim paras to move to the label-based approach, at the moment we are still using position-based

We want to set up an Anaconda package which then allows for collaborative work on colab.google as this requires the installation of soepy.

Please refactor by using https://docs.scipy.org/doc/numpy/reference/generated/numpy.fill_diagonal.html instead.

Please ensure that all integrations are also set up with GITHUB so that our PR's are tested with all our quality indicators i.e. travis, codacy, etc

Integrate use of ose_utils library for in testing harness.

This is specific to the model, so why is it included in the specification file? What happens if we specify a different value?

it seems this is quite similar to the inner construct_covariates functionality. Can we refactor and align the treatment?

Please set up the __init__.py to provide the package's public functions properly by adding, for example,

from soepy.python.simulate.simulate_python import simulate
allowing direct access to soepy.simulate.

https://github.com/simonw/sqlite-utils/blob/master/tests/test_black.py

We want to revisit the assignment of levels of education based on the years of education the individuals in the data have completed.

Note:
In order to speed up regression tests and ensure that robustness tests are running properly at the same time, we reduced the years of education in the simulated data from 14 to 12. For this value, the only education levels now observed in the data frame is low [1,0,0].

@boryana-ilieva created some doctests, @SeBecker please find out and implement their test in our testing workflow. https://docs.python.org/2/library/doctest.html

The current coverage rate of the master branch is only 83%. This seems to be driven by the relatively low coverage of the shared_auxiliary.py and the solve_auxiliary.py modules. In addition the Codecov batch does not reflect the current coverage rate of the project (It seems like Codecov hasn't run an analysis since 1 month ago.).

We would like to achieve a coverage rate above 90% (green batch). For this purpose we should add new tests to our test framework or adjust already existing ones.

We want to make sure that, at every point in time, the current version of the code can simulate based on the example init files included in doc/tutorial/.

PACKAGE_DIR points one level higher than soepy and at the root of the repository. I would like us to not reference anything outside package itself there. So, instead of

PACKAGE_DIR = Path(__file__).parent.parent.absolute() TEST_RESOURCES_DIR = PACKAGE_DIR / "soepy" / "test" / "resources"

we should use

PACKAGE_DIR = Path(__file__).parent.absolute() TEST_RESOURCES_DIR = PACKAGE_DIR / "test" / "resources"
However, this requires to check how the tests that use the two paths are affected.

Please add the capabilities for robustness tests to our development test setup. We need a simple script that allows sending random model specs to our SOEPY code for simulation for a given amount of time, i.e. 1 hour. Here is our setup in https://github.com/OpenSourceEconomics/respy/blob/janosg/development/testing/robustness/run_robustness.py. However, this can be much simpler in our setup …

As we want (if we want) to distribute our package we will use anaconda https://anaconda.org/opensourceeconomics . So we can remove all files that are simply there for PYPI. This is probably the case for setup.py.

Please follow https://numpydoc.readthedocs.io/en/latest/format.html at least in our core functions ...

============================================================================================= warnings summary =============================================================================================
/home/peisenha/.local/share/anaconda3/lib/python3.7/site-packages/matplotlib/rcsetup.py:20: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Iterable, Mapping

/home/peisenha/.local/share/anaconda3/lib/python3.7/site-packages/matplotlib/colors.py:53: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Sized

/home/peisenha/.local/share/anaconda3/lib/python3.7/site-packages/yaml/constructor.py:126: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
if not isinstance(key, collections.Hashable):

-- Docs: https://docs.pytest.org/en/latest/warnings.html

Why is it still here? https://github.com/OpenSourceEconomics/soepy/tree/types/doc/tutorial

We want to develop a nice and strict setup for testing the user input so in exchange do much less testing further down the code by using immutable objects wherever possible. In addition, we want to ensure informative error messages for the user input as well

This can just be replaced by ...

states_all, states_number_period, mapping_states_index, max_states_period = state_space_args

This was bugging me all day today, so here is a Pythonic solution.

attrs = ['seed_sim', 'shocks_cov', 'num_periods', 'num_agents_sim']
draws_sim = draw_disturbances(*[getattr(model_params, attr) for attr in attrs])

This will make sure we always have our environment for the documentation build set up properly going forward.

Can we please have horizontal entries, so the label is clearly mapped to the value?

Fedor suggested a more memory-efficient setup for storing the state space elements. We could consider moving there as well to preempt future problems. This issue serves as a reminder.

Review the need to generate new regression vault after refactor.
To Dos:

• adjust data frame to account for benefits
• change assignment of level of education according to new reduced range of years of education in the simulated sample.

We want to have an A from the automated code review. Some of the raised issues (for example security) are not relevant for us but these can then be removed from inspection through proper codacy configuration.

We want to run all our OSE projects through an organizational account on codacy. Currently, these appear to be linked to individual accounts https://app.codacy.com/project/SeBecker/soepy/dashboard

Please configure environment.yml for our inspections of the dependencies. Otherwise, it always looks in setup.py and throws numerous warnings https://www.jetbrains.com/help/pycharm/managing-dependencies.html

We want to ensure that code quality is tested using pre-commit hook.

This needs to be educ_min, right?

... so this is not an issue anymore.

/Desktop/soepy$ conda env create -f environment.yml
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you.
Collecting package metadata: done

This code

can be easily replaced by

    for group in init_dict.keys():
        for label in init_dict[group].keys():
            setattr(model_params, label,  init_dict[group][label])

Please add blank lines between the different groups of imports, see https://www.python.org/dev/peps/pep-0008/#imports

Please integrate pycharm's basic code inspection in your worfkflow, there are serval unused arguments passed into function or local variables never used.

Please remove all hard-coded numbers from the package and instead just specify them in the config file,

We want to extend the test to all columns. It remains a good idea to store the whole data frame. We will see if we need to cut down on the sample sizes and run into problems with file storage.

Document the new form of the state space, the backward induction and the simulation procedure. Introduce the changes due to numba and model extensions types and kids as version 0.2

We do not want to promote our documentation at this point, please remove badge in README.md

We need a proper conda environment file and .envrc added to the repository as well.

There is no need for us to maintain the package in the anaconda cloud and it results in failing builds now https://travis-ci.org/OpenSourceEconomics/soepy/builds/584675349. So, let us just go ahead and remove it everywhere.

There is no need for this distinction as we will always use numba instead of any FORTRAN extensions.

Please add to README.md

Please restructure the content of the __init__.py`` files so that the command in the header executes the pytest library and also add this call to the the travis script. Please also move the soepy_config.py`one level up.

What is this? Can we turn it off?

Review https://marshmallow.readthedocs.io for a proper setup of our read / write process of the model parameterizations.

We want to improve our code coverage, see https://codecov.io/gh/OpenSourceEconomics/soepy