Code Monkey home page Code Monkey logo

odm2 / odmtoolspython Goto Github PK

View Code? Open in Web Editor NEW
31.0 33.0 15.0 152.09 MB

ODMTools is a python application for managing observational data using the Observations Data Model. ODMTools allows you to query, visualize, and edit data stored in an Observations Data Model (ODM) database.ODMTools was originally developed as part of the CUAHSI Hydrologic Information System.

License: BSD 3-Clause "New" or "Revised" License

Python 2.91% Shell 0.01% Batchfile 0.01% Inno Setup 0.02% PLpgSQL 97.04% PowerShell 0.01%

odmtoolspython's Introduction

ODMToolsPython

Get started by visiting the [ODM Tools Documentation] (https://github.com/ODM2/ODMToolsPython/wiki/ODMTools-Python-Documentation) page.

ODMTools is a python application for managing observational data using the Observations Data Model. ODMTools allows you to query, visualize, and edit data stored in an Observations Data Model (ODM) database. ODMTools was originally developed as part of the CUAHSI Hydrologic Information System.

If you want to try ODM Tools, we have created a couple of testing databases that you can use.

ODM Compatibility

ODM Tools Python is currenlty fully compatible with ODM Version 1.1.1. We are working on ODM2 compatibility, and should have it worked out this winter.

Releases

To make running ODMTools easier we have included installers. Please select the one that is appropriate for your operating system

Windows

Recommended Release:

Mac

Linux

  • Please run from source

Running From Source

Install the following software/libraries. It is recommended to create an environment using 'Conda'. 'Conda' comes prepackaged in the Python Scientific Package Suite Anaconda (Available for Windows, OSX, Linux)

Example environment creation:

  • conda create -n odmtools python=2.7.8 wxpython matplotlib pandas numpy scipy pyodbc sqlalchemy pip psycopg2

  • pip install pymysql

if psycopg2 isn't found, download it manually and follow these directions

Once you have downloaded the source code and all the dependencies installed, run the main application:

python ODMTools.py

if installing pyodbc onto a Mac machine please follow these instructions:

pyodbc Configuration Download the source code ​here​. Unzip the directory where ever you want. Edit ​setup.py​ within the pyodbc directory. Look for the line that says “OS/X now ships with iODBC.” Right underneath that, remove the letter i from the statement:

settings[‘libraries’].append(‘iodbc’)

so that it reads:

settings[‘libraries’].append(‘odbc’)

Now that the setup.py file is configured correctly, pyodbc is ready to be installed. From your virtual environment, do a pip install of pyodbc and use the modified package.

pip install -e PATH_TO_PYODBC

Sponsors

This project is receiving or has received support from a number of agencies and complementary efforts including:

iUTAH    CUAHSI    NSF

This material is based on work supported by the National Science Foundation Grants IIA-1208732, ACI-1339834, and EAR-1224638. Additional support was previously provided by National Science Foundation grant EAR-0622374. Any opinions, findings, and conclusions or recommendations expressed are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

odmtoolspython's People

Contributors

ambersjones avatar denvaar avatar horsburgh avatar jamesyp avatar jmeline avatar kwuz avatar sreeder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

odmtoolspython's Issues

Data Filter Selection

At times, the data filter selects points that do not meet the criteria of the filter. The screen shot below shows points that were selected with a filter for a change value of 5. The change between the points is obviously less than 5.

image

data Filter radio button automatic selection

In 'filter points', when filtering out points for selection the radio button should move automatically based on where the user types in a value.
i.e. if they put in that the value should be greater than 5 the radio button should move to there and filter by greater than without the user having to move the radio button.

create updater

create a script built into ODM Tools to check and see if there is a more recently released version on github and download it to replace the src folder.

Plot/Series Selector Clear/Refresh Button

There needs to be some sort of clear and/or refresh button for plotting and series selection. Perhaps this is what the "refresh" button in the Date Time menu is for, but it is not functioning.

For example, when a new filter is applied and series are selected, it plots them as multiple time series. To remove a series from the plot, the user needs to apply the proper filter and then de-select it. This particular issue might be addressed by a new table or frame that allows the user to toggle plotted series on and off.

I notice that that program slows and gets buggy after repeated series selection/de-selection (and sometimes just using a Simple Filter). Some sort of reset functionality might overcome this.

https://hydroserver.codeplex.com/workitem/10181

Create variable scripts a get command.

When saving a series, if the user creates a variable or a method the scripter only prints a 'get' statement. What if the script is being run against a database where the variable/method does not exist? the get command will cause an error.

Title/Axes Overlap

For box and whisker or histogram plots with multiple series, the titles overlap the axes labels regardless of re-sizing.

image

image

Zoom to Selected

It would be extremely helpful to have some sort of 'zoom to selected' functionality. It is difficult to find selected points on the plot. Another function that would be really helpful would be the ability to right-click on a point in the table and have 'Zoom To Point' be an option.

Clean up imports in Gui

Add class definitions to init.py in src/Gui so imports are cleaner.

This will allow changing lines like

data_filter = frmDataFilters.frmDataFilter(self, self.parent.getRecordService())

to

data_filter = frmDataFilter(self, self.parent.getRecordService())

Data table sorting

Should the table be able to be sorted by any of the columns. or should we leave it as being sorted by datetime at all times

Data series will not plot

There are several data series that will not plot at all. So far it seems to be series with MethodDescription of "Water chemistry grab sample collected by technicians in the field" (e.g., SeriesID 30, SeriesID 28, among others). The Python window returns:

sqlite3.IntegrityError: columns DataValue, LocalDateTime, SiteID, VariableID, MethodID, SourceID, QualityControlLevelID are not unique

Note that several of these series have the same SiteID, VariableID, MethodID, QualityControlLevelID, but different MethodID values.

Points selected for script execution

How do I know which points (if any) are selected for the recording and execution of the scripts? Which steps were "restored"? Which points were interpolated? Which points were deleted?

image

Memory and speed

code is slow and get out of memory error when plotting series with an extremely large number of data points
ideas: pandas and pytables.

Refactor memory_db.py

The in-memory database can be refactored to use SQLAlchemy and resemble the services.

Confirm Delete Dialog

When "Delete Point" is executed, there should be some sort of confirmation dialog, such as "Are you sure you want to delete the selected points?"

Series Selection

In using the Simple Filter for series selection, I can select a Site, but often the Variable selection is grayed out and I am unable to select a Variable. This sometimes occurs after a site has been selected, but it always occurs when no site is selected and only the Variable filter box is selected, so I effectively can't select by variable alone.

https://hydroserver.codeplex.com/workitem/10178

Lasso Selct and Scripting

currently the lasso only prints a comment '#lasso' to the script console. it should print the list of datetimes that are being selected.

Simple Filter

The variable and site selector on the Simple Filter sometimes selects data series outside of the selected parameters (e.g., simple filters for "USU-LBR-Paradise-Little Bear River at McMurdy Hollow near Paradise, Utah" and "USU6-Turbidity" also selects data series at Little Bear River Paradise Weather Station and data series with variables that include 6 such as 62, 63, 64, etc).

Plot axis resize

If a series has been selected for plotting, then de-selected and another data series selected, the x-axis range does not re-size itself to the date range of the newly selected data series. This seems to sometimes occur for the y-axis range as well.

Qualifiers Not In Table

When flags are added to points, they do not show up in the table as qualifiers, so it is not immediately obvious that the flag function is working.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.