chris-ha458 Goto Github PK

oscar-tools

The original tooling for the OSCAR corpus rewritten in Rust

oscar-website

The website of the Oscar Project

palm-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

pes2o

Pretraining Efficiently on S2ORC!

pii-transform

Perform transformations on PII instances detected in documents

pytorch-image-models

PyTorch image models, scripts, pretrained weights -- (SE)ResNet/ResNeXT, DPN, EfficientNet, MixNet, MobileNet-V3/V2/V1, MNASNet, Single-Path NAS, FBNet, and more

pytorch-lightning-image-models

a pytorch lightning based repo of the venerable pytorch lightning image models repo

regex

An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs.

res2net-pretrainedmodels

(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper "Res2Net: A New Multi-scale Backbone Architecture"

rfcs

RFCs for changes to Rust

rust-bloom-filter

A fast Bloom filter implementation in Rust

rust-hashmap-overhead

sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

sonic-rs

A fast Rust JSON library based on SIMD.

subset_sum

Solves subset sum problem and returns a set of decomposed integers.

suffix

Fast suffix arrays for Rust (with Unicode support).

symphonia

Pure Rust multimedia format demuxing, tag reading, and audio decoding library

text-dedup

All-in-one text de-duplication

tlsh

xxh enhanced version of Rust port of TLSH

tokenizer_exp

tokenizer_manipulations

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

txlsh

tlsh with pyo3 and soon xxhash

ungoliant

:spider: The pipeline for the OSCAR corpus

unicode_names2

char <-> Unicode character name (maintained fork of huonw/unicode_names)

vers

very efficient rank and select

vocapxlm

Code for EMNLP2021 paper "Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training"

warc-specifications

Centralised repository for WARC usage specifications.

wyhash-rs

wyhash fast portable non-cryptographic hashing algorithm and random number generator in Rust

xxhash-c-sys

Rust raw bindings to xxHash