This is a Streamlit (in Snowflake) app that provides basic live data ingestion stats.
Here's a preview:
Pre-requisites:
- Any table in Snowflake that data is being ingested into - batch or streaming. Good examples are in this Snowflake Quickstart: Tour of Ingest
Steps to implement:
-
Copy + paste the code from
sis-app.py
to a new Streamlit app in your Snowflake account (on Snowsight) -
Run the app
-
Within the app:
- Choose the table of choice to observe
- Turn on auto-refresh at the bottom of the app and choose a refresh interval of choice. Best choice is to keep it same as the parameter
buffer.flush.time
in your kafka configuration