• StreamStats version: 0.1.5
• Python version: 3.8.11
• Operating System: Windows 10

Description

I was using streamstats to get different basin characteristics in watersheds in GA. If I understand correctly, there is a function within the package that determines the state the lat/long coordinate is in so it can make an API call with that rcode. However, I think that the rcode technically coincides with the streamgrid that the lat/long coordinate is near.

See the stream grids here: https://streamstatsags.cr.usgs.gov/StreamGrids/directoryBrowsing.asp

What I Did

There was a particular lat/long coordinate that falls within Florida, but the streamgrid for Georgia actually extends down into this area of Florida.

To confirm this I tried running the coordinates through the StreamStats Service Documentation (https://streamstats.usgs.gov/docs/streamstatsservices/#/).

I tried both of the following REST Query URLs:
https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=FL&xlocation=-82.12233534&ylocation=30.26887137&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true

and:
https://streamstats.usgs.gov/streamstatsservices/watershed.geojson?rcode=GA&xlocation=-82.12233534&ylocation=30.26887137&crs=4326&includeparameters=false&includeflowtypes=false&includefeatures=true&simplify=true

I found that the GA rcode garners a response but the FL one gives an error. In my mind I think one solution would be downloading all the stream grids for each state and then creating a polygon for each of those based on the extents. Those polygons would essentially be the states you would want those lat/long coordinates fall into and determine the rcode based off of.

As a test I ran the following code:

lat, lon = 30.43632736, -82.28711271
ws = streamstats.Watershed(lat=lat, lon=lon)
print(ws)

I got the following error:

RetryError: HTTPSConnectionPool(host='streamstats.usgs.gov', port=443): Max retries exceeded with url: /streamstatsservices/watershed.geojson?rcode=FL&xlocation=-82.28711271&ylocation=30.43632736&crs=4326&includeparameters=True&includeflowtypes=False&includefeatures=True&simplify=False (Caused by ResponseError('too many 500 error responses'))

Please disregard or correct me if I am incorrect!

Currently there is no automated way of knowing whether links in the documentation are broken, but Sphinx has some awesome functionality through linkcheck that can automatically check for broken external links.

This can be run from the docs/ directory in the terminal via

make linkcheck

or from the project directory via

make -C docs linkcheck

To make this run any time Travis CI builds the project, we need to add make -C docs linkcheck to the docs test environment in our tox.ini file. So instead of the current line here: https://github.com/earthlab/streamstats/blob/master/tox.ini#L30, we need something (I think) more like

[testenv:docs]
commands = 
    make docs
    make -C docs linkcheck

which should cause docs to build and links to be checked when running tox from the command line, and when building the project on Travis CI.

• Add make -C docs linkcheck to tox.ini in the docs environment
• Test locally by running tox from the home directory of this project to make sure linkcheck runs
• Submit a pull request and verify that Sphinx checks all external links when building the documentation

@nkorinek you will need to a dd the sphinx gallery to this setup in order to add the little vignettes.
See earthpy for examples:
https://github.com/earthlab/earthpy/blob/main/docs/conf.py#L41

you then will want to update the dev reqs file with those extensions - look at earthpy or matplotcheck for how this is setup. get in touch with any issues

Currently the tests for streamstats hit the Open Street Map (Nominatim) and USGS servers, so that the tests could fail due to failures in the servers or the code. It would be better know that failing tests are due to the streamstats codebase, and not these servers.

• Mock OSM's Nominatim API for tests associated with streamstats.utils.find_address()
• Mock USGS's StreamStats APi for tests associated with streamstats.watershed.Watershed

The available_flow_stats() method no longer works in this package, or in the interactive StreamStats web services site. Maybe this is related to the gov't shut down...

e.g., https://travis-ci.org/earthlab/streamstats/jobs/455633764

Currently watchdog is a depdendency for streamstats that enables the documentation to be served and edited simultaneously. That's not so important to me, so I'm going to remove that functionality and remove watchdog as a dependency.

Both the Watershed.available_characteristics() and Watershed.get_characteristics() methods currently raise not implemented errors.

The available_characteristics() method

The available_characteristics() method should have no parameters (other than self) and return a dictionary of key value pairs, where they keys are parameter codes and the values are parameter names, e.g.,

from streamstats import Watershed
shed = Watershed(40.009631, -105.242433)
shed.available_characteristics()

{'BSLDEM10M': 'Mean Basin Slope from 10m DEM',
 'CSL1085LFP': 'Stream Slope 10 and 85 Longest Flow Path',
 'DRNAREA': 'Drainage Area',
 'EL7500': 'Percent above 7500 ft',
 'ELEV': 'Mean Basin Elevation',
 'ELEVMAX': 'Maximum Basin Elevation',
 'I24H100Y': '24 Hour 100 Year Precipitation',
 'I24H2Y': '24 Hour 2 Year Precipitation',
 'I6H100Y': '6 Hour 100 Year Precipitation',
 'I6H2Y': '6 Hour 2 Year Precipitation',
 'LAT_OUT': 'Latitude of Basin Outlet',
 'LC11BARE': 'LC11BARE',
 'LC11CRPHAY': 'Percent_Crops_and_Hay_from_NLCD2011',
 'LC11DEV': 'Percent Developed from NLCD2011',
 'LC11FOREST': 'LC11FOREST',
 'LC11GRASS': 'LC11GRASS',
 'LC11IMP': 'Percent_Impervious_NLCD2011',
 'LC11SHRUB': 'LC11SHRUB',
 'LC11SNOIC': 'Percent snow and ice from NLCD 2011 class 12',
 'LC11WATER': 'LC11WATER',
 'LC11WETLND': 'LC11WETLND',
 'LFPLENGTH': 'LFP length',
 'LONG_OUT': 'Longitude of Basin Outlet',
 'MINBELEV': 'Minimum Basin Elevation',
 'OUTLETELEV': 'Elevation of Gage',
 'PRECIP': 'Mean Annual Precipitation',
 'RCN': 'Runoff-Curve Number',
 'RUNCO_CO': 'Soil Runoff Coefficient',
 'SSURGOA': 'SSURGO Percent Hydrologic Soil Type A',
 'SSURGOB': 'SSURGO Percent Hydrologic Soil Type B',
 'SSURGOC': 'SSURGO Percent Hydrologic Soil Type C',
 'SSURGOD': 'SSURGO Percent Hydrologic Soil Type D',
 'STATSCLAY': 'STATSGO Percentage of Clay Soils',
 'STORNHD': 'Percent Storage from NHD',
 'TOC': 'Time of concentration in hours'}

The Watershed class creates a list of watershed parameters upon initialization, accessed via:

shed.data['parameters']

There is one element per parameter, where each element is a dictionary:

shed.data['parameters'][0]

{
  "ID": 0,
  "name": "Mean Basin Slope from 10m DEM",
  "description": "Mean basin slope computed from 10 m DEM",
  "code": "BSLDEM10M",
  "unit": "percent",
  "value": 2.44
}

To make a dictionary of code/name pairs to be returned by available_parameters() you could do something like:

dict((p['code'], p['name']) for p in shed.data['parameters'])

The get_characteristics() method

This method should return a dictionary of data associated with any valid parameter code, e.g.,

shed.get_characteristics('BSLDEM10M')

{
  "ID": 0,
  "name": "Mean Basin Slope from 10m DEM",
  "description": "Mean basin slope computed from 10 m DEM",
  "code": "BSLDEM10M",
  "unit": "percent",
  "value": 2.44
}

Or, provided no particular parameter code, return an error with a message telling users that they need to provide a parameter code (and that a list of available parameter codes can be found with the available_characteristics() method).

Implementation checklist

• Write unit tests for available_characteristics() and get_characteristics()
• Write docstrings for both methods
• Link to the USGS site that describes each parameter in the docstring for available_characteristics(): https://streamstatsags.cr.usgs.gov/ss_defs/basin_char_defs.aspx
• Implement these methods

Currently, if a user makes a query with a lat/lon point that returns no results for the location, they get a ValueError that states that no results were returned. It would be better to use string substitution to expand on this error message to print out the latitude and longitude values that were used.

Users get this message when a call to utils.find_address finds nothing at the lat/lon location: "No results found!" https://github.com/earthlab/streamstats/blob/master/streamstats/utils.py#L19

To get this message, try the following:

from streamstats import utils
utils.find_address(lat=40, lon=-37)

But, it would be better to get a message like "No results were found for the point (lat=40, lon=-37). Are you sure this point is in the United States?". This might help to catch errors if users mix up the lat and lon arguments, or if they don't realize that points need to be in the U.S.

• Programmatically generate a better error message to pass to ValueError in streamstats/utils.py
• Delete the comment # make this better on line 19
• Write a test case to make sure the error message contains the lat and lon values in the right order, and any other desired text in tests/test_utils.py

• StreamStats version:
• Python version:
• Operating System:

Description

I was trying to get the GEOJSON for a watershed

What I Did

I executed this script in Python 27

import streamstats
from streamstats import Watershed
Watershed (lat=39.9644,lon=-106.5392 )

The coordinate cited above is on a stream reach in Colorado.  I got the exact grid from the StreamStats application for a test.

After running the script I got an error, telling me that the point must be in the US (50 states).

Traceback (most recent call last):
  File "C:\Users\cmoser\Desktop\watershed.py", line 6, in <module>
    Watershed(lat=39.9644,lon=-106.5392)
  File "D:\Python27\ArcGIS10.4\lib\site-packages\streamstats\watershed.py", line 30, in __init__
    self.state = utils.find_state(utils.find_address(lat=lat, lon=lon))
  File "D:\Python27\ArcGIS10.4\lib\site-packages\streamstats\utils.py", line 55, in find_address
    assert address['country'] == 'USA', 'Point must be in US (50 states)'
AssertionError: Point must be in US (50 states)

Need to add @seilerman, @jdiaz4302, @NateMietk, and @lwasser as authors in AUTHORS.rst.

Currently the get_geojson() method raises a not implemented error. This method should return the string GeoJSON representation of the watershed boundary. The Watershed class already collects this data at the initialization step, and stores it in the data attribute, which is a dictionary:

import json
from streamstats import Watershed
shed = Watershed(40.009631, -105.242433)

# extract geojson from watershed data
watershed_boundary = shed.data['featurecollection'][1]['feature']

# print the geojson in a nicely formatted string
print(json.dumps(watershed_boundary, indent=2))

Note that this package has not added json as a dependency yet, so this will need to be done to implement this method.

• Write unit tests for get_geojson to verify that it returns the expected data
• Implement Watershed.get_geojson() using the information above

Both Watershed.available_flow_stats() and Watershed.get_flow_stats() currently return not implemented errors. Implementing these will be a bit harder than some other features, because it will require additional calls to the StreamStats API. See https://streamstats.usgs.gov/docs/streamstatsservices/#/ > Flow Statistics for more information.

The API calls will need the two letter state code and a workspace ID (contained in the attribute data['workspaceID']), e.g.,

https://streamstats.usgs.gov/streamstatsservices/flowstatistics.json?rcode=NY&workspaceID=NY20181030172158127000&includeflowtypes=true

The Watershed.available_flow_stats() should take no arguments other than self, and return a dictionary of parameter code/name pairs.

The Watershed.get_flow_stats() can return the JSON data returned by the API call to compute flow statistics.

Note:

Not all workspace IDs return valid flow statistics. For example, the following API call for the Colorado workspace ID containing the SEEC building returns a 500 (internal server error) code:

https://streamstats.usgs.gov/streamstatsservices/flowstatistics.json?rcode=CO&workspaceID=CO20181030164518210000&includeflowtypes=true

• Write unit tests for both methods
• Document both methods
• Implement both methods
• Gracefully handle 500 response codes from the server

• StreamStats version: 0.1.4
• Python version: 2.7.13 and 3.6.9
• Operating System: Windows 10 Enterprise

Description

I'm trying to generate scenario flow reports for a number of basins. I'm particularly interested in 25, 50, and 100-year peak flow discharge rates.

What I Did

Everything currently works for me as far as pulling basin characteristics and delineating watershed - would just like added functionality