machinelearningsystem Goto Github PK

nvidia_sgemm_practice

Step-by-step optimization of CUDA SGEMM

ok-topk

Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.

omnireduce

ooo-backprop

oort

Oort: Efficient Federated Learning via Guided Participant Selection

openmlsys-zh

《机器学习系统：设计与实现》

optimizing-sgemm-on-nvidia-turing-gpus

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

optimizing-sgemv-on-nvidia-gpus

An implementation of SGEMV with performance comparable to cuBLAS.

optimus-cc

[ASPLOS'23] Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression

osdi21_ae

Artifact for OSDI'21 GNNAdvisor

osdi21_ae-gnn

Artifact for OSDI'21 GNNAdvisor

osdi22-graphminer

Graph Pattern Mining

osdi22-unity

A distributed deep learning framework that supports flexible parallelization strategies.

osdi23-brainstorm

osdi23-hydro

Surrogate-based Hyperparameter Tuning System

osdi23-mms

Multi model serving

pagraph

SoCC'20 and TPDS'21: Scaling GNN Training on Large Graphs via Computation-aware Caching and Partitioning.

parallel-sgd

一个用于联邦学习或分布式机器学习，对不同网络拓扑进行实验的简易框架。

parity-models

Learning-Based Coded Computation

patrickstar

PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP. Democratize AI for everyone.

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

pet-osdi21-ae

petps

PetPS: Supporting Huge Embedding Models with Persistent Memory

pets-atc-2022

The Artifact of PetS (ATC'22)

pilotfish

PilotFish harvests the free GPU cycles of cloud gaming with deep learning training

pipedream

pipelearn

Pipeline parallelism in distributed machine learning

piper

pipeswitch

PipeSwitch: Fast Pipelined Context Switching for Deep Learning Applications

pipetransformer

PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021