Mkdocs pages crash. Need to investigate why.

Use the wiki - eg for instructions on setting up the notes

More on style guides/PEPs

PEPs (Python Enhancement Proposals) are design documents for Python. They are primarily aimed at developers of features in the language, but some also contain clear definitions of how to use the language well. The most important of these is PEP8, but PEP257 is also of relevance.

On top of these core language definitions, there are ‘house styles’ for particular organisations or code bases, to ensure a consistent style of code, comments etc to both developers and users.

Further, there are many documents with suggestions on how to write good code, clear comments etc. since these are things you will be marked on, you should pay some attention to these.

Full list of PEPs

https://www.python.org/dev/peps/

Most relevant PEPs

doc strings:

https://www.python.org/dev/peps/pep-0257/

###code style: pep8
https://www.python.org/dev/peps/pep-0008/

Further pep8 advice:

https://realpython.com/python-pep8/

house styles

https://google.github.io/styleguide/pyguide.htm

comments

https://realpython.com/python-comments-guide/

make README.md and notebooks/README.md consistent

Dear Colleague,

The MODIS Collection 6.1 Snow Cover and Sea Ice data sets, accessible through the NASA National Snow and Ice Data Center Distributed Active Archive Center (NSIDC DAAC), were released in the last year and data reprocessing is now complete. The full temporal coverage, from February 2000 for MODIS/Terra data sets and July 2002 for MODIS/Aqua data sets to present, is available. Collection 6.1, also known as Version 61, is largely compatible with Collection 6/Version 6, but NSIDC urges users to switch to the newer Collection 6.1 as Collection 6 will stop forward processing in mid 2022.

Access to the data sets and their documentation is provided at https://nsidc.org/data/modis/data_summaries

Data Acknowledgments
Data Authors: Hall, D. K. and G. A. Riggs
Data center: NSIDC DAAC (https://nsidc.org/daac)
Sponsor: NASA EOSDIS (https://earthdata.nasa.gov)

If you have any questions please contact the User Services Office at [email protected].

We use np.prod to get array size for multi d grids. Could simplify with using size.

Should add size to when shape is introduced in numpy.

equations not rendering in markdown eg
https://ucl-eo.github.io/geog0111/042_Weighted_smoothing_and_interpolation/

"In convolution, we combine a signal (y) with a filter (f) to achieve a filtered signal. "

The python script lut_RMSE.py requires executable permission to be called in 050_Models.ipynb Exercise 1.

$ chmod +x geog0111/notebooks/geog0111/lut_RMSE.py

In the model answer, the function to acquire MODIS data is updated to geog0111.modisUtils. modisAnnual.

Need to remove or process update fields in md files for mkdocs

In the geog0111/notebook/018_Running_Python.ipynb, the first code cell under the section Running a Python script got a warning of Permission denided because the file geog0111/helloWorld.py is not executable.

!geog0111/helloWorld.py
/bin/bash: geog0111/helloWorld.py: Permission denied

!ls -l geog0111/helloWorld.py
-rw-r--r-- 1 ucfawya ucfapb 514 Oct 6 17:01 helloWorld.py

The possible solutions to this would be:
Method 1: make the file executable
!chmod u+x geog0111/helloWorld.py

Or

Method 2: call python interpreter
!python geog0111/helloWorld.py

fix the docker image

In Exercise 3 in Notebook 015 Control in Python: if

line 16: month = 'out of bounds error: doy='+doy

The last variable 'doy' is integer and cannot be concatenated with string. The solution would be converting 'doy' into string type:
month = 'out of bounds error: doy=' + str(doy)

Before 'Formative assessment', notes ended without introducing how to feed sys.argv into helloWorld.py

tidy up old directory

Location: The Summary section at the end of the notebook 040_GDAL_mosaicing_and_masking.ipynb.

An update might be required for the last column in the row of gdal.BuilVRT():
replace file, sds = modis.get_files(year, doy) with

data = getModisFile(**kwargs)
for sds, sds_v in data.items():
      for doy, doy_v in sds_v.items():
            files = doy_v.keys()
            ofile = f"work/stitch_{sds}_{kwargs['year']}_{doy:03d}_{'Tiles_'+'_'.join(tiles)}.vrt"
            stitch_vrt = gdal.BuildVRT(ofile, list(doy_v.values()))
            del stitch_vrt

Sphinx version broken for rtd- change to using mkdocs in config file. Once mkdocs issues resolved

separate out clog and other main codes

In the notebook 070_Summary.ipynb, please update 021 URLs:

from geog0111.gurlpath import URL -> from urlpath import URL

Update advice to students on lambda functions wrt https://www.python.org/dev/peps/pep-0008/

specifically, don’t define a lambda function as variable unless warranted. Show use of lambda fn in eg pandas map function

Course basics 1
Put earthdata in here
Submit running python

Core concepts 2,3
Submit code

Datasets 4,5

Coursework part 1

Image processing 6,7
Submit code
Geospatial data 8,9
Submit code

Coursework part 2

This would leave 1 week for slippage.

Need to have implicit for loops in for section. These are used later on but not introduced.

Notebook 023 Plotting:
In the markdown cell under the section 'NOAA precipitation', the url link for the file 2276931.csv is not accessible any more.

In the notebook 050_Models.ipynb, some typos might cause confusion and would be better to update:

Prerequisties section:

• np**/mgrid -> np.**mgrid
• "You should by know be familiar" -> "You should now be familiar"

Models section:

• the last line above the first code cell: "This has two parameters p[0] and p[0]" -> the second p[0] might be p[1]

Discrepancy and optimisation section:

• "The best approach with depend on circumstances" -> "The best approach will depend on circumstances"

problem running exercise 5.

I need to look inside the URL library for why this fails.

--> parsing URLs from html file 1 items
--> discovered 1 files with pattern .h08v06.hdf in https://e4ftl01.cr.usgs.gov/MOTA/MCD15A3H.006/2020.06.01
--> trying https://e4ftl01.cr.usgs.gov/MOTA/MCD15A3H.006/2020.06.01/MCD15A3H.A2020153.h08v06.006.2020160231732.hdf
MCD15A3H.A2020153.h08v06.006.2020160231732.hdf : -0.00 MB
--> code 401
--> failed to connect

Given that MOSIS will soon finish collecting data, the examples and c/w should probably be re-developed fro some other data source. Something with Sentinel-2 time series would be interesting.

A simpler job would be to migrate to VIIRS, which no doubt has a similar snow and LAI product.

I leave this for the next convenor of the course to manage.

need to revisit and fix the docker setup for the course

Unreadable Notebook: /nfs/cfs/home3/Uucfa6/ucfalew/geog0111/notebooks/021_Streams.ipynb NotJSONError('Notebook does not appear to be JSON: '{\n "cells": [\n {\n "cell_type": "m...')****

I think you (qingling) have edited the file but broken the formatting -- can you roll it back and fix?

Notebook 023_Plotting.ipynb - the example data in Errorbar Section have NAN values for 2022 Oct-Dec because there is no data for these months yet.

The error bar graph cannot be plotted correctly unless get rid of nan in std. So, I would suggest to replace:

mean = np.array(df[months]).mean(axis=1)
std  = np.array(df[months]).std(axis=1)

by:

mean = np.nanmean(np.array(df[months]), axis=1)
std  = np.nanstd(np.array(df[months]), axis=1)

tidy s/w in geog0111

make sure each session has full summaries of what has been learned

• I need to finish redoing the last session (fitting phenology model based on Jose’s previous)

• I need to sort out what material to use for feedback assessments

• I need to decide what the new submitted practical will be
  There is opportunity for a new part 1 submitted practical at half term. I was thinking submit some code … a bunch of functions and tests to do XXX
  The 2nd part will be snow still, but changed in form

• fix some known bugs:

  • nasa login test broken and links on pages broken
    DONE
  • Fire dataset image missing from repo
    DONE
  • sort documentation script
    TESTING
    bin/notebook-mkdocs.sh

Move packages notes to the intro

Trying to install geog0111 on os-x M1 computer.

Error msg on running:

mamba env create -y --force -n geog0111 -f environment.yml
is

Could not solve for environment specs
The following package could not be installed
└─ pyephem does not exist (perhaps a typo or a missing channel).

I seem to recall that pyephem is being depreciated. We use it in some of the practicals (also for geog0133?) so they should;d be checked for this dependency and updated. It'll be ok for the Ucl notebooks, as it's all installed there. May be it's just an M1 issue?