Add configuration file for Delphi about delphi HOT 14 CLOSED

Added a master_config.json file at the root of the repository that has a key |-> string pair for every path to an external repository or data location required by delphi. In order to not step on each others toes anyone who clones the repo should copy master_config.json into the delphi folder as config.json. After doing this please set the paths enclosed appropriately so that you can access all the external resources as needed.

For reference here is what each of the paths in the config file should point to:

• data_path: This should point to the data directory hosted in the the Program-analysis Google Drive folder
• dssat_repo_path: This should point to your local checkout of the DSSAT repository
• ed2_repo_path: This should point to your local checkout of the Ecosystem Demography Model repository
• clm_repo_path: This should point to your local checkout of the clm portion of the Community Earth System Model repository. Note that the path component to go from the root of cesm/ to clm/ has already been provided in master_config.json

To access these paths I have added a variable config_paths to delphi/paths.py. This loads the JSON file contents as a python dictionary, so the key names provided above can be used to access the specific paths of interest. I have yet to remove the data/ directory from the delphi repository. Once @adarshp and I test this new config file method I will remove the directory.

from delphi.

Changes have been made. Added a new Google Drive shared folder named Delphi/ (path has been added to README), and I changed the config file path variable to delphi_data_path. I still have not removed that data directory from the delphi repo. I am leaving that as a task for @adarshp to complete as soon as he feels comfortable with making the change.

from delphi.

Yep, we can execute scripts on Travis ... I'll let you know if I hit a dead end.

from delphi.

@adarshp Can we get a list going of all the external resources we expect regular delphi users to need to have access to? I imagine that we might need two config files, one for delphi-only users and one for program-analysis users as well that want access to PA resources. So far all I can think of for PA is a pointer to the DSSAT resource. Are there any others?

from delphi.

Here are a few things that I can think of:

• Indicator data for parameterizing models (right now, is at data/south_sudan_data.csv)
• The FORTRAN models that David LeBauer pointed us to.
• Gradable adjective data (right now, at data/adjectiveData.tsv
• Concept-to-indicator mapping (right now, at data/concept_to_indicator_mapping.txt)

from delphi.

This is great, thank you.

Two suggestions:
(1) rename data_path to "program_analysis_data" (or pa_data)
(2) add these instructions with paths (i.e., the links to how to get the data) to the README.md (I'm not doing this edit myself on the chance that either of you think otherwise).

from delphi.

@cl4yton the data directory that this points to holds more than just data for program-analysis, and as I am typing this I'm realizing that it should probably be hosted in a delphi-specific Google Drive folder instead of the one intended specifically for program analysis. Perhaps we could make a Google Drive shared folder for delphi and add this data folder there? We could then change this path variable to delphi_data_path. Also I do want the "_path" to be included in the name so that it's clear that this variable is accessing a directory path and not the actual data itself.

from delphi.

@adarshp Ah, got it, that helps clarify. Then I do think keeping it with "delphi_data" rather than the more specific "program_analysis_data" makes sense. I also agree that we should add "_path" to the end -- I like having global variable names indicate what type they represent. And I also agree that we should make a Google Drive shared folder for delphi and add this data there.

from delphi.

Thanks @pauldhein - still need to test a couple of things, I'll remove the data directory after that.

from delphi.

@pauldhein : Any reason why you suggest doing

cp master_config.json delphi/config.json

instead of

cp master_config.json config.json

?

Also, I don't think we necessarily need a separate master_config.json and this copying step - Eidos just has a single eidos.conf file that they suggest changing to activate specific options.

The other reason why I think it would be good to have a single file that users can modify would be to get Travis CI to run properly - right now it's complaining about not being able to find a config.json file - if we tailor the contents of the config file (which should perhaps be named delphi_config.json rather than config.json, to be more specific) to the relevant locations for Travis, we can get the automated tests to pass.

from delphi.

@adarshp : Correct me if I'm missing something, but I think the rationale @pauldhein is following for making a copy is that otherwise everyone will have to dirty their repo with their locally edited version of the master, and there's danger of the local copy being committed. In Paul's scheme, we copy to a local version that is .gitignore'd and so we can have our local copy, protect the master, and not dirty our local repo. At least that's my understanding.

from delphi.

Okay, got it - let's go with the copying method then, and I will think of a workaround for Travis.

from delphi.

The Travis issue is... and issue. And Travis is important. That's possibly why Eidos goes that route. Let us know if you're hitting a dead-end -- don't want this to become a rabbit hole. Is there a way to have a Travis config that, say, executes a script?

from delphi.

Closed by #89 and #90 .

from delphi.