c-tc Goto Github PK

calculon

cookbook

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

dace

DaCe - Data Centric Parallel Programming

deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

dhs-llm-workshop

DHS 2023 LLM Workshop by Sourab Mangrulkar

flash-attention

Fast and memory-efficient exact attention

json-tutorial

从零开始的 JSON 库教程

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

lpg2vec

master_thesis

megablocks

megatron-llm

distributed trainer for LLMs

megatron-lm

Ongoing research training transformer models at scale

mlir-dace

Data-Centric MLIR dialect

ms-amp

Microsoft Automatic Mixed Precision Library

nanotron

Minimalistic large language model 3D-parallelism training

nccl

Optimized primitives for collective multi-GPU communication

nccl-tests

NCCL Tests

notes_semester_1

olmo

Modeling, training, eval, and inference code for OLMo

oneflow

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

optimizations-of-ball-arithmetic

pspin-benchmark-sparse-reduction

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

reliable-transport-project

sponge

CS144 Lab Assignments

taco

The Tensor Algebra Compiler (taco) computes sparse tensor expressions on CPUs and GPUs

torchtitan

A native PyTorch Library for large model training

transformerengine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

triton

Development repository for the Triton language and compiler