bjmsong Goto Github PK

bjmsong.github.io

caffe

Caffe: a fast open framework for deep learning.

concurrency-in-python-with-asyncio

Code for the Manning book Concurrency in Python with Asyncio

cuda_programming

Code from the "CUDA Crash Course" YouTube series by CoffeeBeforeArch

cudalibrarysamples

CUDA Library Samples

cutlass

CUDA Templates for Linear Algebra Subroutines

deeplearningfromscratch

《深度学习入门：基于Python的理论与实现》电子版及配套代码​。​

deepspeedexamples

Example models using DeepSpeed

dlrm

An implementation of a deep learning recommendation model (DLRM)

fastllm

纯c++的全平台llm加速库，支持python调用，chatglm-6B级模型单卡可达10000+token / s，支持glm, llama, moss基座，手机端流畅运行

flash_attention_inference

Performance of the C++ interface of flash attention, flash attention v2 and self quantized decoding attention in large language model (LLM) inference scenarios.

hands-on-gemm

Matrix Multiplication Optimization on GPU/CPU

hw05

inference_results_v3.1

This repository contains the results and code for the MLPerf™ Inference v3.1 benchmark.

lectures

Material for cuda-mode lectures

libflash_attn

Standalone Flash Attention v2 kernel without libtorch dependency

llama2.c

Inference Llama 2 in one file of pure C

llm.c

LLM training in simple, raw C/CUDA

micrograd

A tiny scalar-valued autograd engine and a neural net library on top of it with PyTorch-like API

mlperf_inference_vllm

nanogpt

The simplest, fastest repository for training/finetuning medium-sized GPTs.

tensorrt_pro

C++ library based on tensorrt integration

triton

Development repository for the Triton language and compiler

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

xllm

A lightweight llama2 inference framework. It can inference llama2-7b with 166+ tokens/s on signle 4090.