flytigerw Goto Github PK

base-quantization

base quantization methods including: QAT, PTQ, per_channel, per_tensor, dorefa, lsq, adaround, omse, Histogram, bias_correction.etc

cnn-quantization

Quantization of Convolutional Neural networks.

computer

计算机类知识

cuda_sgemm

cudabmk

Source for Demystifying GPU Microarchitecture through Microbenchmarking

cute-gemm

cutlass-kernels

cutlass_fpa_intb_gemm

deepgemm

dfq

PyTorch implementation of Data Free Quantization Through Weight Equalization and Bias Correction.

dissectingtensorcores

embedded-ai.bench

benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.

evt_ae

Artifacts of EVT ASPLOS'24

exercises

daily exercise

gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

gptq-for-llama

4 bits quantization of LLaMa using GPTQ

gpu-arch-microbenchmark

Dissecting NVIDIA GPU Architecture

igemmgen

Code generator for implicit gemm algorithm for amdgpu.

libflash_attn

Standalone Flash Attention v2 kernel without libtorch dependency

llama.cpp

Port of Facebook's LLaMA model in C/C++

llm-awq

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

lsqplus

LSQ+ or LSQplus

megpeak

micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regula

mmyolo_tensorrt

mperf

mperf是一个面向移动/嵌入式平台的算子性能调优工具箱

mrecg

outlier_suppression

The official PyTorch implementation of the NeurIPS2022 paper, Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models

outlier_suppression_plus

Quantization of large language models

owq

Code for the "OWQ: Lessons learned from activation outliers for weight quantization in large language models".