River Charts is a Python, Django, Plotly, and Pandas web application that visualizes river data for a specific river/site/location.

• The line graph is driven by data pulled using an API from the United States Geological Survey (USGS).
  • The data is captured by the USGS using a gage height sensor every 15 minutes.
• The data is updated with the most recent river height data every time the application is loaded.
  • Please be patient, the loading time of the application is around 60+ seconds (depending on your internet connection), due to the large API JSON response and the amount of data being processed from the USGS.
    • As a future enhancement (or if you want to fork/develop it), I'd like to cache the data on the server, or store it in a database with a scheduled data refresh.
    • I'd also like to investigate using Websockets for real time updates.
    • This will allow the removal of the loading/skeleton screen for the initial load/wait time due to the API request that is sent out during every page visit/load.
• I recommend using the application on a desktop since the chart is interactive has a wider view, but it can be used on a mobile device as well.
• Visit the application here.

• Features
• Background Story
• Definitions
• Getting Started
  • Dependencies
  • Configuration
  • Running Locally
  • Deployment
• What's Inside?
• API Documentation
  • Calling the API
  • API Output Example
• Disclaimer
• What's Next?
• Project
• Contributing
• Resources
  • Technical Resources
  • Educational Resources
• License
• Credit

• Responsive: The application is responsive and can be used on a mobile device.
• Interactive: The application uses Plotly to create an interactive line graph.
• Data Visualization: The application visualizes river data for a specific river/site/location.
• Data Source: The application sources data using an API that returns JSON output.
• Data Processing: The application processes the JSON output and converts it to a Pandas DataFrame.
• Data Manipulation: The application manipulates the DataFrame to create a Plotly line graph.
• Data Update: The application updates the data with the most recent river height data every time the application is loaded.
• Data Capture: The data is captured by the USGS using a gage height sensor every 15 minutes.

Every year, my friends and I float 2 miles down the Susquehanna River in NEPA on river tubes (a 2 hour float). I wanted to create a web application that would allow us to visualize past river data in order to see the river height on the days we floated down the river. Some float dates, we still got together, but we didn't float due to the dangerous river levels. Green plots on the graph represent the dates we did float (with a "Floated" status of "Yes"), and Red plots represent the dates we did not float (with a "Floated" status of "No").

The closest river gauge to our float location is the Susquehanna River at Meshoppen, PA, which is the default site code in the application. You can change the site code in the application to visualize data for a different river.

West Branch, Susquehanna River.

The application uses gage height data to plot the river height in feet on a given date.

Here are some definitions to help you understand the terminology used in this document:

• USGS: The United States Geological Survey. The USGS is a science organization that provides impartial information on the health of our ecosystems and environment, the natural hazards that threaten us, the natural resources we rely on, the impacts of climate and land-use change, and the core science systems that help us provide timely, relevant, and useable information.
  • USGS
• Gage Height: The height of the water surface above the gage datum (zero point). Gage height is often used interchangeably with the more general term, stage, although gage height is more appropriate when used with a gage reading. Stage is more appropriate when used with a recorded or calculated gage height.
  • Gage Height

This project makes use of several libraries and frameworks:

• Python: For the application logic.
• Django: For web application functionality.
• Plotly: For creating interactive visualizations.
• Pandas: For data manipulation and analysis.
• Requests: For making API calls.
• Python-Decouple: For storing sensitive information in a .env file.
• See requirements.txt for a full list of dependencies.

• Edit config.py to add your own USGS API (and other) information.
  • USGS API
  • USGS API Documentation
• Toggle USE_DUMMY_DATA to True in config.py to use dummy data instead of the API.
  • This is useful for testing the application without making API calls.
• The float data plots are driven from a .csv file located in static/data/river_charts.csv.
  • This file can be edited to add/remove float dates.
  • The file is read in views.py and passed to the template as a context variable.

To install and run the project locally, follow the steps below:

1. Clone the repository:

   git clone https://github.com/scottgriv/River-Charts

   • Cloning a repository

2. Navigate to the project directory:

   cd [YOUR PROJECT DIRECTORY]

3. Create a virtual environment:

   python -m venv venv

   • Creating Virtual Environments

4. Activate the virtual environment:

   source venv/bin/activate

   • Activating a virtual environment

5. Install the required packages using requirements.txt:

   pip install -r requirements.txt

   • Installing Packages

   Note: If you wany to generate a new requirements.txt file, run the following command:

   pip freeze > requirements.txt

   • Requirements Files

6. Run the Django server:

   python manage.py runserver

   • Running the Django server

Now, you can visit http://127.0.0.1:8000/ in your browser to access the application.

• The application is hosted here on PythonAnywhere.
• The application is deployed using a WSGI configuration file.
  • WSGI Configuration
• First, make sure you adjust your settings.py file ALLOWED_HOSTS to include your deployment host.
  • Deploying Django
• Second, make sure you adjust your settings.py file DEBUG to False for production.
  • Django Settings
• Finnaly, be sure to create a .env file where you host your application to store your sensitive information (excluded from this repository).
  • Python-Decouple

Below is a list of the main files and folders in this repository and their specific purposes:

River-Charts # Root folder
├─ config.py # A file that contains sensitive information (excluded from this repository).
├─ manage.py # A command-line utility that lets you interact with this Django project in various ways.
├─ requirements.txt # A list of Python packages required to run this project.
├─ River_Charts # A directory for the river_charts app.
│   ├─ __init__.py # An empty file that tells Python that this directory should be considered a Python package.
│   ├─ admin.py # A file that registers models to be displayed in the Django admin site.
│   ├─ apps.py # A file that contains the application configuration.
│   ├─ models.py # A file that contains the database models.
│   ├─ tests.py # A file that contains the tests for the application.
│   ├─ urls.py # A file that contains the URL declarations for the application.
│   └─ views.py # A file that contains the application logic.
├─ river_charts # The Django project directory.
│   ├─ __init__.py # An empty file that tells Python that this directory should be considered a Python package.
│   ├─ asgi.py # An entry-point for ASGI-compatible web servers to serve your project.
│   ├─ settings.py # Settings/configuration for this Django project.
│   ├─ urls.py # The URL declarations for this Django project.
│   └─ wsgi.py # An entry-point for WSGI-compatible web servers to serve your project.
├─ static # A directory for static files that are used in this Django project.
│   ├─ css # A directory for CSS files.
│   │   └─ styles.css # A CSS file that contains the styles for the application.
│   ├─ data # A directory for data files.
│   │   └─ river_charts.csv # A CSV file that contains the float dates for the application.
│   └─ images # A directory for image files.
├─ templates # A directory for HTML templates.
│   └─ river_charts # A directory for HTML templates specific to the river_charts app.
│       ├─ error.html # An HTML template that displays an error message.
│       └─ index.html # An HTML template that displays the application.
├─ views.py # A file that contains the application logic.
├─ .github # GitHub folder
├─ .gitignore # Git ignore file
├─ .gitattributes # Git attributes file
├─ PRG.md # PRG Connection File
├─ LICENSE # A file that contains the license for this project.
└─ README.md # This file.

Below is the documentation for the API used in this application.

To call the API and retrieve the data:

1. Make a GET request to: http://nwis.waterservices.usgs.gov/nwis/... (based on your requirements).

   • ex. https://waterservices.usgs.gov/nwis/iv?format=json&sites=01533400&startDT=2015-07-01&endDT=2023-08-16&parameterCd=00065&siteStatus=active&siteType=ST

2. Pass the necessary parameters in the request.

   • ex.

   params = {
       "format": "json", # Set your interchange format.
       "sites": "01533400", # Site Code: Susquehanna River at Meshoppen, PA.
       "startDT": "2015-07-01", # Set the date you want to start collecting data from.
       "endDT": "2023-09-14", # This is based on the current date in the application.
       "parameterCd": "00065", # Parameter Code: Gage height, ft.
       "siteStatus": "active", # Selects sites based on whether or not they are currently active. Each USGS Water Science Center determines whether a site is active or inactive. The default is all (show both active and inactive sites).
       "siteType": "ST", # ST = A body of running water moving under gravity flow in a defined channel. The channel may be entirely natural, or altered by engineering practices through straightening, dredging, and (or) lining. An entirely artificial channel should be qualified with the "canal" or "ditch" secondary site type.
   }

   • USGS Site Web Service
     • Format
     • Sites
       • Site Example
     • Parameter Codes
       • List of Parameter Codes
     • Site Status
     • Site Type
       • List of valid Site Types
   • Codes and Parameters

3. Process the JSON response as demonstrated in the example above.

Notes:

• The application uses the API to source data for the graph. If the API is down, the graph will not render.
• Errors are handled in the application by redirecting the user to an error page with the appropriate error message.
  • HTTP errors are handled in the application by redirecting the user to an error page with the appropriate error message.
  • The application will also display an error message if the API returns an empty response or a timeout error (default is set to 90 seconds in config.py).
  • The API is rate limited to 30 calls per minute. If you exceed this limit, you will receive a 429 error.

Example of an Error Page.

The application sources data using an API that returns JSON output. Here's an example of what the API response looks like:

{
    "name": "ns1:timeSeriesResponseType",
    ...
    "timeSeries": [
        {
            "sourceInfo": {
                "siteName": "Susquehanna River at Meshoppen, PA",
                ...
            },
            "variable": {
                "variableName": "Gage height, ft",
                ...
            },
            "values": [
                {
                    "value": [
                        {
                            "value": "15.13",
                            "dateTime": "2015-07-01T00:00:00.000-04:00"
                        },
                        ...
                    ]
                }
            ]
        }
    ]
}

(For the sake of brevity, the full output is abbreviated with ...)

• The data provided by this application is sourced from the USGS.
• It's subject to revision, and for more information, please refer to their official disclaimer.
• Software is provided as-is and no warranty is given about its usability.

Thank you for taking the time to read through this document and I hope you find it useful! If you have any questions or suggestions, please feel free to reach out to me.

Please reference the SUPPORT file in this repository for more details.

I'm looking forward to seeing how this project evolves over time and how it can help others with their GitHub Portfolio.

Please reference the CHANGELOG file in this repository for more details.

Please reference the GitHub Project tab inside this repository to get a good understanding of where I'm currently at with the overall project.

• Issues and Enhancements will also be tracked there as well.

Feel free to submit a pull request if you find any issues or have any suggestions on how to improve this project. You can also open an issue with the tag "bug" or "enhancement".

• How to contribute:

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/River-Charts)
3. Commit your Changes (git commit -m 'Add new feature')
4. Push to the Branch (git push origin feature/River-Charts)
5. Open a Pull Request

Please reference the CONTRIBUTING file in this repository for more details.

Below are some external resources I found helpful when creating River Charts:

• Python - A programming language that lets you work quickly and integrate systems more effectively.
• Django - A high-level Python Web framework that encourages rapid development and clean, pragmatic design.
• Plotly - A Python graphing library that makes interactive, publication-quality graphs online.
• Pandas - A fast, powerful, flexible and easy to use open source data analysis and manipulation tool.
• Requests - A simple, yet elegant HTTP library.
• Python-Decouple - A Python library for separating the settings of your Django/Flask/FastAPI project from the source code.
• Deploying Django - A tutorial on how to deploy Django.
• Django Settings - A list of all settings available in Django.
• Running the Django server - A tutorial on how to run the Django server.
• Creating Virtual Environments - A tutorial on how to create virtual environments.
• Activating a virtual environment - A tutorial on how to activate a virtual environment.
• Installing Packages - A tutorial on how to install packages using pip.
• Requirements Files - A file containing a list of items to be installed using pip install like so: pip install -r requirements.txt.
• PythonAnywhere - A Python hosting provider with a free tier.

• USGS - The United States Geological Survey.
• USGS API - The USGS Instantaneous Values Web Service.
• USGS API Documentation - The USGS Instantaneous Values Web Service Documentation.
• USGS Site Web Service - The USGS Site Web Service.
• Codes and Parameters - The USGS Codes and Parameters Documentation.
• How to Get Real-time Flood Data from the USGS API - A tutorial on how to get real-time flood data from the USGS API.
• USGS Client Library - Client library for interfacing with USGS datasets (GitHub repository).
• USGS Client Library Repository - Client library for interfacing with USGS datasets.
• USGS API Homepage - The USGS API Homepage.
• USGS Rest API web services - The USGS Rest API web services.
• USGS Instantaneous Water Service data URL generator - The USGS Instantaneous Water Service data URL generator.
• National Water Dashboard - The National Water Dashboard.
• Rivers vs. Streams vs. Creeks (USGS) - A USGS article on the difference between rivers, streams, and creeks.
• How to Make Colored Country Maps with GeoPandas and Python - A tutorial on how to make colored country maps with GeoPandas and Python.
• 5 Ways to Use Free Data on the Internet for Science - A tutorial on how to use free data on the internet for science.
• GitHub for Biologists - A tutorial on how to use GitHub for biologists.

This project is released under the terms of the GNU General Public License, version 3 (GPLv3).

• The GPLv3 is a "copyleft" license, ensuring that derivatives of the software remain open source and under the GPL.
• For more details and to understand all requirements and conditions, see the LICENSE file in this repository.

Author: Scott Grivner
Email: [email protected]
Website: scottgrivner.dev
Reference: Main Branch