Code Monkey home page Code Monkey logo

site's Introduction

Automating GIS-processes

This repository contains the source files for the course Automating GIS processes II at the University of Helsinki. While the course is taught as an on-site lab course for students of the Masters’ programme in geography there, anybody interested in learning how to use Python to automate the handling and analysis of geographic information is welcome to follow the course content at autogis-site.readthedocs.io.

License and terms of usage

Creative Commons License

We hope that the teaching material provided here is helpful to other teachers and learners. Thus, we share all the lessons openly, as well as the source code and sample data.

Teaching material and code snippets are licensed under the Creative Commons Attribution-ShareAlike 4.0 International licence and the GNU GPLv3 license, respectively.

Read more about the license and terms of usage here.

Browser based programming environment

The instructions in this course provide a programming environment that allows anyone to immediately try and run all sample code directly in the browser. This functionality is realised using Binder and the CSC’s Notebooks (the latter are available to students and staff at Finnish universities, only).

Instructions for updating the University of Helsinki course each year

There a few tasks that need to be done to prepare this course for being taught each autumn. Even without updating any of the content, reserve at minimum a few days to get everything set up.

  1. Before you do anything else, make sure to tag the previous year’s course content, so that readthedocs provides a yearly version. This can (and should) have been done at the end of last autumn’s course, but please double-check:

    git clone [email protected]:Automating-GIS-processes/site
    cd site
    git tag -a 2022  # <- CHANGE THE YEAR!
    git push --tags
    
  2. Update the Python environment (ensure stable versions throughout the lifetime of the course and of the course page’s version for the current year)

    • Update the pinned versions in pip’s requirements.txt (rtd uses pip!):

      # create a new virtual environment and activate it
      python -m venv --clear .virtualenv
      source .virtualenv/bin/activate
      
      # install using the _unpinned_ docs/requirements.in.txt
      pip install -r docs/requirements.in.txt
      
      # save a pinned docs/requirement.txt
      pip freeze --local -r docs/requirements.in.txt > docs/requirements.txt
      
      # deactivate the virtual environment
      deactivate
      
    • Update conda’s environment.yml in a similar way:

      • First, make sure environment.in.yml pins Python (the interpreter itself) to the lastest major version available from conda-forge. For instance, in autumn 2022, the version listed is 3.10.6; accordingly, the respective line in the environment file should read - python=3.10.
      • Then use the following commands (they’re analogue to the pip workflow, above):
        # remove a possibly existing stale environment
        conda env remove --name=autogis
        
        # install a new environment using the _unpinned_
        # ci/environment.in.yml
        conda env create --file=environment.in.yml
        
        # save a pinned vi/environment.yml
        # (removing the ‘prefix:’ line because it hard-codes
        # your system’s path)
        conda env export --no-builds --name autogis \
        | grep -Ev '^prefix:' \
        > environment.yml
        

      Note, that Python’s release schedule typically foresees a new Python version around the time of the start of the course. For instance, 2022, Python 3.11 was released 7 days before the course started. Most likely you want to stick with the previous version, as some of the packages might not have been updated in time.

  3. Update the docker image for the CSC Notebooks, and upload it to CSC’s registry. Create a CSC Notebooks ‘workspace’ and ‘application’ using the docker image. Follow the instructions in the Automating-GIS-processes/csc-notebook-dockerfile repository

  4. Set up Slack: Create a new Slack instance, e.g., ‘Automating-GIS-processes-2023’, and set up the channel structure such as in previous years: #general and #random exist by default, create #week1 through #week7, as well as #final-assignments; create #instructors to coordinate with the student tutors. I (@christophfink) will leave the 2022 version around, drop me a line and I’ll add you there so you can see for what we used Slack.

  5. Set up GitHub classroom: register for GitHub Education (https://classroom.github.com), create a new organisation for this year’s exercise repository templates (see Automating-GIS-processes-2022, and add the exercise repositories there, one by one, as the course advances. For each exercise, create a classroom, and update the links in the respective docs/lesson-X/exercise-X.md files.

  6. Update the course details defined in conf.py. Things that change every year are set up as ‘substitutions’, variables that are replaced throughout all documents. Edit them to reflect this year’s course’s details (as of October 2022, this block is starting at line 42 of conf.py):

    # The following are the main things that need to be updated every year
    # These variables are replaced throughout the course documents
    # (see the comment for the first item for an example)
    myst_substitutions = {
        "year": "2022",  # use {{year}} in markdown files to replace it with the current value
        "starting_date": "Tuesday, 1 November 2022",
        "lectures_weekday_time_location": "Tuesday, 15:15-16:45, Exactum C222",
        "work_sessions_weekday_time_location": "Friday, 12:15-16:00, Physicum A113+A114 (GIS lab)",
        ...
    }
    

Contact

This course was conceived and designed by Henrikki Tenkanen, and has been taught, maintained and improved over the years by Vuokko Heikinheimo and Håvard Wallin Aagesen. Currently (autumn term 2022), Christoph Fink is the responsible teacher.

You can find contact information on the course website.

site's People

Contributors

christophfink avatar htenkanen avatar kamyar68 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

site's Issues

Reconsider the format (the material is ideal for a flipped classroom setting)

The way the lessons are prepared (very detailed, and definitely working 95% of the time) makes teaching them in classroom meetings slightly redundant. The time in the classroom could be used more productively, e.g., by revisiting and reapplying the topics worked through independently at home. Ultimately this would mean having students study and follow what’s now the ‘lessons’ as homework, and converting the current ‘weekly exercises’ into tasks to be carried out in pairs or small groups during the classroom meetings.

This is just an idea, and would entail other consequences: e.g., it would question the need for student tutors, or at least the separate meeting with them.

Lesson 3: AttributeError: 'GeoSeries' object has no attribute 'get_values'

The function get_nearest_values() contains the following line:

nearest_value = nearest_data[value_column].get_values()[0]

which fails with the above error.

The method get_values() is deprecated since Pandas version 0.25.0. You can use DataFrame.values instead.

I'm using Pandas v1.0.4 and GeoPandas v0.7.0.

Working code:

nearest_value = nearest_data[value_column].values[0]

Error while reading shapefile with geopandas

my commands are as follows:

import geopandas as gp
fl=r"M:/rathore/vic_5km/L2_data/L2_data/DAMSELFISH_distributions.shp"
data=gp.read_file(Path('M:/rathore/vic_5km/L2_data/L2_data/DAMSELFISH_distributions.shp').resolve())

The 3rd commands gives the following error:

TypeError: invalid path: UnparsedPath(path='M:/rathore/vic_5km/L2_data/L2_data/DAMSELFISH_distributions.shp')

I have tried to change the / to \\ but its still there.

Update exercise 6

Exercise 6 has not been updated since 2018-ish, and it’s showing.

Recreate final assignment and make it more versatile

Currently, the final assignment offers a lot of detailed guidelines for two of the three suggested topics. However,

  1. the third topic is completely hanging in the air
  2. the two more defined topics are really close to each other, it might be good to include example cases from other areas in the students’ studies
  3. the general instructions are not very detailed (e.g., some students in 2022 initially thought they had to complete all three options)
  4. the assignments should be less constrained, i.e., appeal more to the creative application of the newly-learnt skills rather than provide a step-by-step cooking recipe

ad 2) there could be more topic suggestions, for instance, touching upon remote sensing or physical geography example cases

Clean the data repository

The ./data repository contains quite large files that would be good to clean. However, check that doing that does not break anything in terms of building the materials.

Exercise 4 is missing

Going through all the lessons now, and unfortunately the github page for Exercise 4 is missing so I am unable to do it.

Lesson 2: read/write different file formats - missing files

I'm not able to find anywhere the files "Finland.*" in multiple formats (i.e. tab, gdb, gpkg, kml, geojson, shp). I've looked not only under L2_data folder, but everywhere. I've even looked under the 2018 course's data folders.

The file is also missing if I run the notebook in Binder.

Do I need to download it from somewhere else?

Thanks!

Automating-GIS-Processess. Lesson 3

Hi.

In Lesson 3 of Automating-GIS-Processes: nearest-neighbour.ipynb - (def get_nearest_values_) you might consider replacing .get_values()[0] with .to_numpy()[0].

This is a great resource.
Thank you.
Adrian.

distance computations in L3 nearest neighbor

I think there is a problem with the computation of distances in L3 nearest neighbor function due to a mix up of lat/lng order.

For example, if you look at the output 7 in nearest-neighbor-faster.ipynb :

the distance between POINT (24.85584 60.20727) and POINT (24.85745 60.20749) is 180.521584

when I try to replicate with haversine or sklearn , I get 92 meters instead.

I also tried switching lat/lng pairs, then you get the 180.521584 meter output. But my understanding is that the correct distance should be 92m.

Update lesson 6

Lesson 6 has not been updated in autumn 2022 with all other lessons. That means, it is still using string-mangling to create file paths (instead of pathlib.Path), its overarching story line is not streamlined very well, and it generally is one generation behind the other lessons in terms of optimising, proofreading and the like

Would be good to focus on it for autumn 2023

Include the ‘extra’ lessons, again

In autumn 2022, I did not find the time to update the two ‘extra’ lessons (see 2021 edition), that cover PyQGIS and raster handling, to the new format. This should be included again, or possibly incorporated into the regular lessons

Lesson 2: printing all available Drivers supported by GDAL

Using the below fiona/GDAL releases:

fiona.__version__
-> '1.8.13'

fiona.gdal_version
-> GDALVersion(major=3, minor=0, revision=4)

I get the below error when I try to import GDALEnv:

from fiona._drivers import GDALEnv
-> ModuleNotFoundError: No module named 'fiona._drivers'

I'm able to get the same output by doing the following:

from fiona.env import GDALEnv
env = GDALEnv()
env.drivers().keys()

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.