Ecallisto NG is a compact yet effective Python package designed to facilitate seamless interaction with the Ecallisto API (Rest API built with FastAPI). It provides a simple interface to fetch data from the API, as well as some basic data processing and plotting capabilities. Behind the Rest API, there is a TimeScaleDB database, which is a time-series database. Thus, data access by time, with aggregation functions, is fast and possible.
The package is constructed in Python 3.9 and utilizes the requests
library to directly access the Ecallisto Rest API via the link: https://v000792.fhnw.ch/api/redoc.
Currently, not all data is added to the database. Feel free to reach out if you need data before 2020.
To install this package, clone this repository and use pip for installation. Execute the following command in your terminal:
pip install -e .
For this, you need to install PyTorch.
Ecallisto NG is conveniently available on PyPI as well. To download, visit the following link: https://pypi.org/project/ecallisto-ng/
Please have a look at the jupyter notebook under example
. ¨
Here's a guide on how to use the different features of Ecallisto NG. Be aware that data is cached automatically in your working directory under ecallisto_ng_cache
. Feel free to delete this folder if you want to fetch the data again.
Fetching data is easy using the get_data
function, housed under the ecallisto_ng.data_fetching.get_data
module. Here's an example:
from ecallisto_ng.data_fetching.get_data import get_data
parameters = {
"instrument_name": "austria_unigraz_01",
"start_datetime": "2021-03-01 06:30:00",
"end_datetime": "2021-03-07 23:30:00",
"timebucket": "15m",
"agg_function": "MAX",
}
df = get_data(**parameters)
You can also check the availability of data using the get_tables
and get_table_names_with_data_between_dates
function, housed under the ecallisto_ng.data_fetching.get_information
module. Here's an example:
from ecallisto_ng.data_fetching.get_information import get_tables, get_table_names_with_data_between_dates
from datetime import datetime, timedelta
get_tables()[:5]
get_table_names_with_data_between_dates(
start_datetime=(datetime.now() - timedelta(hours=24)).strftime("%Y-%m-%d %H:%M:%S"),
end_datetime=datetime.now().strftime("%Y-%m-%d %H:%M:%S")
)
Ecallisto NG provides basic plotting capabilities. Here's an example of how to generate a spectogram:
from ecallisto_ng.plotting.utils import fill_missing_timesteps_with_nan, plot_spectogram
df_filled = fill_missing_timesteps_with_nan(df)
plot_spectogram(df_filled, parameters["instrument_name"], parameters["start_datetime"], parameters["end_datetime"])
We also provide some basic functionalities to edit the spectogram. Here's how you can do it:
from ecallisto_ng.data_processing.utils import elimwrongchannels, subtract_constant_background, subtract_rolling_background
df = elimwrongchannels(df)
df = fill_missing_timesteps_with_nan(df)
df = subtract_constant_background(df)
df = subtract_rolling_background(df)
plot_spectogram(df, parameters["instrument_name"], parameters["start_datetime"], parameters["end_datetime"])
These simple commands allow you to easily manipulate spectogram data, enabling effective use of the Ecallisto API for your needs.
Please have a look at the example-notebooks under example
for more information.