Code Monkey home page Code Monkey logo

lyzer-etl's Introduction

Lyzer-ETL

lyzer-etl's People

Contributors

evanlab02 avatar pranav-zodes-44 avatar github-actions[bot] avatar

Stargazers

 avatar  avatar

Watchers

 avatar

lyzer-etl's Issues

Setup Auto Updater using Github API

Description

We would like to have our app auto-update when new releases are released to Github, this will download the assets/artifacts so that we can have easier time updating our app.

Acceptance Criteria

  • Downloads new release artifacts when releases are made.
  • Attempts to, where possible to replace the current binary/executable with the new one.
  • Restrict users from performing any commands if they are not on the latest version.

Create Hydrate Schedule Command using Typer

Description

We would like to be able to hydrate a years schedule using this app.

The command would look something like the following.

Lyzer-Hydrate load schedule 2023

Acceptance Criteria

  • Create sub commands file for hydrate commands.
  • Command should retrieve data from ergast for given year.
  • Command should refine data for our database.
  • Command should put refined data into the database.
  • Command should reject invalid years.
  • (Optional) Print data that was inserted using table print of some sort

Create setup function

Description

We need to create a function that sets up the app with all keys and secrets that will be required in a local config file.

The config file should be stored at {home directory}/.lyzer/hydrate-config.json

Acceptance Criteria

  • Setup retrieves MongoDB connection string from user.
  • Setup saves MongoDB connection string in local config file.
  • Setup ensures MongoDB connection string is given and is valid.

Create TUI for App

Description

It can be convenient to rather have some UI to do hydration instead of just a CLI command. We will use a TUI framework to create a terminal interface that will be able to hydrate data.

Acceptance Criteria

  • Create TUI that can hydrate schedules into the database.
  • Add CLI command that will start Terminal User Interface.

Create Web UI

Description

We would like to also have the chance to use a web interface for hydration if preferred. This should be easy to implement using Streamlit.

Acceptance Criteria

  • Create Web UI for ETL.
  • Add command that starts Web UI.

Create config command

Description

We need a config command that can adjust config settings

Acceptance Criteria

  • Config command should be able to validate and update the MongoDB connection string with given string.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.