Code Monkey home page Code Monkey logo

pycopancore's Introduction

pycopancore

Reference implementation of the copan:CORE World-Earth modelling framework

copan:CORE is developed at the Potsdam Institute for Climate Impact Research.

An extensive documentation is found at https://pik-copan.github.io/pycopancore

Table of Contents:

  1. Introduction
  2. Disclaimer
  3. Quick start guide
  4. Structure of the repository
  5. Licence and Development

Introduction

The pycopancore package is a python implementation of the copan:CORE modeling framework as described in this paper. The framework is designed to allow an easy implementation of World-Earth (i.e., global social-ecological) models by using different model components of environmental, social-metabolic or cultural submodels and combining them with newly developed components. The implementation and simulation of World-Earth models within the framework can use different types of modeling techniques such as differential equations, stochastic and deterministic events and therefore allows to compare different model and component types and implementations.

Reference: J.F. Donges*/J. Heitzig*, W. Barfuss, M. Wiedermann, J.A. Kassel, T. Kittel, J.J. Kolb, T. Kolster, F. Müller-Hansen, I.M. Otto, K.B. Zimmerer, and W. Lucht, Earth system modeling with endogenous and dynamic human societies: the copan:CORE open World-Earth modeling framework, Earth System Dynamics 11, 395–413 (2020), DOI: 10.5194/esd-11-395-2020, * The first two authors share the lead authorship.

pycopancore is developed at the Potsdam Institute for Climate Impact Research (PIK), Research Domains for Earth System Analysis and Complexity Science. Responsible senior scientists at PIK are Jobst Heitzig & Jonathan F. Donges.

Contact: [email protected] Website: www.pik-potsdam.de/copan/software

Disclaimer

This software is provided for free as a beta version still under active development. It has not been completely tested and can therefore not guarantee error-free functioning. Please help us further improving the code by reporting possible bugs via the github issue tracker!

Quick start guide

Installation

For running pycopancore, an installation with python > 3.6 with some additional packages is required.

An easy way to install python is to use the Anaconda environment (https://www.anaconda.com/download/).

The package can be installed by downloading the repository and and running the setup.py script with

$ pip install

from the root directory of the package. The script should automatically install all the required and missing packages.

For developers, the recommended way of installing is to run in the package main directory

$ pip install -e

This creates a link instead of copying the files, so modifications in this directory are modifications in the installed package.

Running a model

To run one of the preconfigured models, execute a python script in the studies folder, for example

python run_seven_dwarfs.py

Documentation

The documentation can be accessed under https://pik-copan.github.io/pycopancore and provides an introduction to the framework, its different entity and process types, a full documentation of the API as well as a step-by-step tutorial.

To create a local html version of the documentation, access the docs directory and type

> make html

The documentation can then be accessed under docs/_build/html/index.html.

To be able to create the automatic UML-Diagrams, pylint and graphviz needs to be installed. To finally create the diagrams, use

> make uml

Code of Good Practice

When contributing to the project, please follow the guidelines below:

  • For every class/function write a proper docstring before committing.
  • Use static values as little as possible. Preferably define a variable in the header of the file instead.
  • For functions describing processes in the model_components, papers from the scientific literature that use these functional forms should be referenced in the code documentation.
  • Use as many assert statements as possible, even if they are computationally expensive. For actual runs, these checks can be switched off using the -O flag of the Python Interpreter.
  • Use proper (and long) variable names. Auto-completion will help typing them.
  • If a similar set of command is used twice, write a function for it right away.
  • Design the metadata used in the Variable class according to established catalogs like the CF conventions.
  • When writing class and method docstrings, already specify types and bounds for arguments and return values in the sphinx-compatible PyContracts way.
  • Follow the "Guidelines for Ensuring Good Scientific Modelling Practice at PIK".

Tests

We are using the python testing framework pytest with pylama for style and error checking. Please write corresponding unittests while developing and make sure that all test pass by executing

py.test

in the root of the project tree.

Requires

  • pytest
  • pylama
  • pylint
  • pylama_pylint
  • pytest-cov, to check of test coverage

Structure of the repository

The code in the repository is organized into different subfolders:

docs contains a detailed description of the framework and scripts to compile API documentation of the code (see Quick start guide/Documentation). Furthermore, it contains the material for the framework tutorials.

examples ??

graphics contains code to visualize model output.

pycopancore contains the main code of the framework, specifically:

pycopancore/data_model implements and specifies the underlying master data model for the different process taxa and provides a base for defining units and keeping them consistent.

pycopancore/model_components contains various subfolders, in which model components and their interfaces with other components are defined.

pycopancore/models contains code files that combine different model components to self-contained models.

pycopancore/private implements classes that are needed for the correct plugging together of model components and other functionality for preparing the model for running.

pycopancore/process_types contains the definitions for the different process types (events, steps, explicit and implicit equations, and ordinary differential equations).

pycopancore/runners contains implementation of model runner, that executes the model by integrating its processes.

pycopancore/util contains auxiliary functions.

studies accommodates the files for executing the models. These "run" files define parameter settings, initialize model entities and start the model runner.

tests comprises code to implement and run testing procedures of the implementation.

Licence and Development

pycopancore is licenced under the BSD 2-Clause License. See the LICENCE file for further information.

The versioning of pycopancore has been chosen to be administrated on the github.com system as the future prospect of this model strongly encourages outside contributions. Any release version will be pushed on the inhouse versioning systems, gitlab and / or svn.

Candidates for speeding up Python code: cython, numba, ...

pycopancore's People

Contributors

fmhansen avatar jakobkolb avatar jdonges avatar kilianzimmerer avatar marcwie avatar mensch72 avatar paulabreitbach avatar sduslu avatar tillkolster avatar wbarfuss avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pycopancore's Issues

Proposed lifecycle and workflow for Model Variable Definitions and Metadata

Goal: Allow for...

  • flexible addition of model variables (including static ones aka parameters) by model components
  • easy usage in other model components
  • possibility to be promoted into a centrally governed "master data model"
  • without having to maintain a variable's metadata (name, description, dimension, unit, scale type, bounds, quantum, default value, uninformed prior distribution, etc.) in more than one place.
  1. A new model variable (e.g. Cell.fossil_reserve) comes into being when some model component (e.g. pycopancore.components.fossil_extraction) introduces it the first time by defining it in their entity interface class by instantiating the Variable class with the variable's metadata, e.g. in pycopancore.components.fossil_extraction.interface.py:
class Cell (object):
    fossil_reserve = Variable(name="cellular fossil reserve", desc="...", dimension=Carbon)
  1. Another components (e.g. pycopancore.components.general_equilibrium_energy_sector) can then access that variable by importing that interface module in its own interface module and assigning the same Variable object to its own entity interface class, e.g. in pycopancore.components.general_equilibrium_energy_sector.interface.py:
import pycopancore.components.fossil_extraction.interface as FE
class Cell (object):
     fossil_reserve = FE.Cell.fossil_reserve
  1. As soon as the variable gets approved by the copancore-community's "data model committee" (to be defined), its definition gets copied into the master data model, a special module of the same format as the components' interface modules, whose classes are however never inherited directly. In the above example, on would then add to pycopancore.master_data_model.py:
class Cell (object):
    fossil_reserve = Variable(name="cellular fossil reserve", desc="...", dimension=Carbon)
  1. From that point on, all components using this variable should import it from the master data model, e.g., both pycopancore.components.fossil_extraction.interface.py and pycopancore.components.general_equilibrium_energy_sector.interface.py would change to
import pycopancore.master_data_model as MDM
class Cell (object):
     fossil_reserve = MDM.Cell.fossil_reserve
  1. To maintain harmonized variable definitions and avoid coincidental name clashes, the method Model.configure() must make sure that if two of the components used in the model at hand define the same variable name, it must point to the same Variable object (as in the examples above). It may even make sure that if a component uses a variable name that exists in the MDM, it must point to the Variable object defined in the MDM.

  2. Still, components should probably be allowed to use another component's (or the master data model's) variables under a locally different name, as in:

import pycopancore.components.fossil_extraction.interface as FE
class Cell (object):
     geological_carbon_stock = FE.Cell.fossil_reserve

ENH: Mandatory Parameter Naming

Maybe include mandatory naming of variables when calling functions. Assure that this is the case for all functions by using the testing environment

Variable naming and metadata standards to use

ENH: Decide on time integration logic and integrator

How does time integration work in the model? What integrator to use? (Frank has an alpha version of a solver of algebro-differential equations)

Basic problem / challenge of combining continuous and irregularly discrete dynamics.

Reducing code complexity

Regarding the McCabe complexity checks, we have to reduce complexity as much as possible and find new complexity thresholds.

improve _template model component

  • rename to _template so that it is not confused with actual model components
  • add __deactivate and __reactivate method templates to all entity types
  • remove as much clutter as possible
  • rewrite docstrings so that they are templates for the later docstrings rather than explain what needs to be done
  • add comments beginning "TODO:" that explain what needs to be done to make a component on the basis of the _template. the model component developer would later remove these comments
  • merge some of the minor improvements Jobst did in the prototype branch

Raising discontinuities during integration

Implement the option to raise events caused by a process during a integration. Until now the time an event takes place is decided before, a sudden event is not possible. Runner-logic must be changed to allow for this.

Plotting component

Write a plotting component which works with the trajectory dictionary which is returned by the runner.

Enhance Templates

Enhance the templates and their documentation to make it easier for newbies to work with the framework. Maybe this is to be done when developing a new component, so nothing is forgotten to be included

PEPs included in py.test

PEP257 is giving out a lot of errors, since it requires a very strict formatting of docstrings. Do we want this? One of the complaints is this:
<<First word of the docstring should not be This [pep257]>>
This might be very annoying during further coding...

General Naming problems

While going through the code Svenja noticed some confusing use of naming. This led us to start a general discussion about it.

Each underlying class (Cell, Society, Individual, Nature, Culture, Metabolism) has an attribute called entities, which is a list containing the classes' instances. Since only objects of the classes Society, Individual and Cell should be called entities we have to find another name for this list.

Any suggestions?

Design MasterDataModel

#25 depends on the implementation of a MasterDataModel, where variables which are used by several components are defined

Draft a first Code of Good Practice for coding

After the meeting, I was discussing with Wolf and he asked me to Draft a first Code of Good Practice for coding, that we could put in the README. The idea is to have some guidelines/recommendations/hints that we want to follow so the code is legible, easy to debug and easy to expand.
PS: I can't assign myself, could somebody with the correct rights do that?

ENH: Define recommended way to handle networks

What is a recommended way to handle networks in CORE components: use a performant, but simple library for this: igraph, graphtool, boostgraph ? Check what is recommended and define criteria for selection.

Modul or Model guided Folder structure?

Either

  • copancore
    • anthroposphere
      • abstract_anthoposphere.py
      • concrete_athroposphere_1.py
      • concrete_athroposphere_2.py
      • ...
    • ecosphere
      • abstract_ecosphere.py
      • concrete_ecosphere_1.py
      • concrete_ecosphere_2.py
      • ...

or

  • copancore
    • abstract_anthoposphere.py
    • abstract_ecosphere.py
    • concrete_model_1
      • concrete_anthroposphere_1.py
      • concrete_ecosphere_1.py
    • concrete_model_2
      • ....
    • ...

Code style/enhancement

If there are any suggestion to improve or enhance the existing code leave them here.

Edit by Till:
This is just a thread to collect pieces of code to show how there is a more elegant way and to keep them as a sort of style guide by examples

World Attribute in Abstract_Entity_Mixin

Jede Entity hat ein Attribut, zu welcher World es gehört. Da das ja bei jeder Entity ausser bei World selber nötig ist, ist das nun im Branch https://github.com/pik-copan/pycopancore/tree/world_entity_type (world_entity_type) in der Abstract_Entity_Mixin Klasse implementiert. Jetzt gibt es nur den Schönheitsfehler, dass man beim Initialisieren von world selber auch etwas angeben muss um dieses Attribut zu füllen, derzeit ist das mit None gemacht. Das ist aber nicht schön. Wie kann man das schöner machen?

create a contributor agreement

From PIK's open source recommendations: "use http://contributoragreements.org to create a contributor agreement". They recommend the following settings:

  • Non-exclusive license: Giving the project owner the right to relicense it but does not retract any rights of the contributor on their contribution
  • Permission to relicense the code under any Open Source license which is either approved by the Free Software Foundation or Open Source Initiative (Option 3).
  • Permission to license provided documentations under cc-by (http://creativecommons.org/licenses/)
  • Traditional patent licensing agreement

Creation Routines

This is a question that came to my mind lately. When initializing the model, where do we have creation routines of cells etc.? At low levels, so for example at the planet or donut_world class that creates a grid or - in case of individuals - a network that includes individuals? Or at model level?

I prefer creating instances at lower level to make the model class more manageable, but I see probels with this idea too: So in my example, the donut_world class is made to work with the local_renewable_resource class. With another class it may not work...

I am not shure how connected these problems are, because I don't have a clear picture of the model structure in detail in my mind, yet. Maybe we can discuss this soon, I think otherwise we will have inconsequent structure later

Unit-test

Since now we are programming a lot in single classes without using them, it might be the right time to write some unit-test? If so, do we put them into the tests folder and name them something like filename_test.py? Or folder_filename_test.py? Is there an established naming scheme?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.