cyanguwa Goto Github PK

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

berkeleygw-gpp-jack-fortran

cocoapi_public_fork

COCO API - Dataset @ http://cocodataset.org/

cracking-the-coding-interview-c--

Cracking the coding interview solutions in c++

deeplearningexamples

Deep Learning Examples

deeplearningprofiling

fairscale

PyTorch extensions for high performance and large scale training.

flash-attention

Fast and memory-efficient exact attention

gpt-neox

An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger.

megatron-lm

Ongoing research training transformer models at scale

nemo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

nersc-roofline

openwebtext

An open clone of the GPT-2 WebText dataset by OpenAI. Still WIP.

system-design-primer

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

test

transformerengine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both training and inference.