Code Monkey home page Code Monkey logo

ml_pyt's Introduction

Pytorch + MLflow + DVC

FashionMNIST Classification with PyTorch

This repository contains an example of training a deep learning model on the FashionMNIST dataset using PyTorch. It demonstrates how to use MLflow for tracking the model and DVC for tracking the data.

1. Install the required packages:

  • pip install -r requirements.txt

2. Data Setup

  • Initialize DVC and set up remote storage (if necessary):
dvc init

Using dvc init in workspace will initialize a DVC project, including the internal .dvc/ directory

  • Configure DVC remote storage (e.g., AWS S3, Google Cloud Storage): this will add data to remote storage
dvc remote add -d remote_storage path/to/your/dvc_remote

dvc add copies the specified directory or files to .dvc/cache or shared_cache/you/specified, creates .dvc files for each tracked folder or file and adds them to .gitignore

  • .dvc and other files are tracked with git add --all
  • Track the data directory with DVC:
dvc add data
dvc push

3. MLflow

MLflow helps in tracking experiments, packaging code into reproducible runs, and sharing and deploying models. You can find more information about MLflow. We have used MLflow to track the experiments and save parameters and metrics used for a particular training. We can include or change parameters according to our requirements Tracked Parameters and Metrics:

  • Epochs
  • Accuracy
  • Loss
3.1 Access the MLflow UI in your browser after running you script:

Run the below command

    mlflow ui

It will host you on the local computer. compare model seeing metrics

3.2 Unmanaged without MLflow CLI

Run the standard main function from the command-line.

python "filename".py <parameter1>
3.3 MLflow CLI - mlflow run

Use the MLproject file. We get more control over an MLflow Project by adding an MLproject file, which is a text file in YAML syntax, to the project’s root directory.

  • mlflow run local
mlflow run "filename" -P <parameter1>=<value>
  • mlflow run github
mlflow run https://github.com/<usernale>/<filename>.git -P <parameter1>=<value>

ml_pyt's People

Contributors

shreshshaurya avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.