This collects and organizes self-contained and complete projects written using the Econ-ARK. The content here should be executable by anyone with a suitably configured computer or using nbreproduce.

Each project lives in its own repository. To make a new REMARK, please see the REMARK starter example.

Types of content include (see below for elaboration):

1. Explorations
   • Use the Econ-ARK/HARK toolkit to demonstrate some set of modeling ideas
2. Replications
   • Attempts to replicate important results of published papers written using other tools
3. Reproductions
   • Code that reproduces ALL of the results of some paper that was originally written using the toolkit

A REMARK refers to an open git repository that is indexed in this repository with appropriate metadata.

The REMARK's repository must:

1. Have a tagged release, the last commit before including it as a REMARK should be tagged with a 1.0 release.
2. In that repository at that release, there must be:

• In either the top-level directory or a binder/ directory, either:
  • installation files for pip:
  • a runtime.txt containing the name of a python version, e.g. python-3.9.0
  • a requirements.txt file with pinned dependencies (such as created by the command pip freeze > requirements.txt), or...
  • installation files for conda:
  • an environment.yml file with pinned dependencies
• A reproduce.sh script that
  • Installs the requirements
  • Runs and reproduces all the results

To index the repository as a REMARK, please file a pull request in this repository adding:

1. A markdown file with all the required metadata about the REMARK. A template for the metadata file with the required fields is in this repository.
   • Note: markdown file's Version field must correspond to the the tag of the release (see above).

It is strongly recommended to include:

• If reproduce.sh takes longer than a few minutes, a reproduce_min.sh that generates some interesting subset of results within a few minutes
• A Jupyter notebook that exposits the material being reproduced.

A maximalist REMARK (the extra stuff is completely optional) includes:

• A reproduce_text-etc.sh that generates the text
• A dashboard that creates interactive versions of interesting figures

In cases where the replication's author is satisfied that the main results of the paper have been successfully replicated, we expect to approve pull requests with minimal review.

We also expect to approve with little review cases where the author has a clear explanation of discrepancies between the paper's published results and the results in the replication attempt.

We are NOT intending this resource to be viewed as an endorsement of the replication; instead, it is a place for it to be posted publicly for other people to see and form judgments on. Nevertheless, pull requests for attempted replications that are unsuccessful for unknown reasons will require a bit more attention from the Econ-ARK staff, which may include contacting the original author(s) to see if they can explain the discrepancies, or may include consulting with experts in the particular area in question.

Replication archives should contain two kinds of content (along with explanatory material): Code that attempts to comprehensively replicate the results of the paper, and a Jupyter notebook that presents at least a minimal set of examples of the use of the code.

This material will all be stored in a directory with some short pithy name (a bibtex citekey might make a good directory name) which, if written in an Econ-ARK compatible style, will also be the name of a module that other users can import and use.

Code archives should contain:

• All information required to get the replication code to run
• An indication of how long that takes on some particular machine

Jupyter notebook(s) should:

• Explain their own content ("This notebook uses the associated replication archive to demonstrate three central results from the paper of [original author]: The consumption function and the distribution of wealth)
• Be usable for someone wanting to explore the replication interactively (so, no cell should take more than a minute or two to execute on a laptop)

The key difference with the contents of the DemARK repo is that REMARKs are allowed to rely on the existence of local files and subdirectories (figures; data) at a predictable filepath relative to the location of the root.