Frank Odom's Projects
Pretrained BLIP with a similar API to CLIP.
PyTorch implementation of BYOL: a fantastically simple method for self-supervised image representation learning with SOTA performance.
Generate text captions for images from their embeddings.
Implementation of the Hindsight Experience Replay (HER) algorithm using PyTorch. Utilizes Deep Deterministic Policy Gradients for off-policy optimization of RL agents in continuous action spaces.
(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307.02486)
RAG application to answer questions about PDF documents using LLMs.
Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.
Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)
Code samples to accompany my "Intro to Deep Learning" article series on Medium.
Simple but robust implementation of LoRA for PyTorch. Compatible with NLP, CV, and other model types. Strongly typed and tested.
Minimalist library for training RL agents in PyTorch. Implements many common training algorithms, with a focus on actor-critic methods. Includes SAC, TD3, PPO, A2C, VPG.
Personal repo recreating Noise Contrastive Score Networks (https://arxiv.org/abs/1907.05600v2) for learning purposes.
Template repo for Python projects, especially those focusing on machine learning and/or deep learning.
Fastest 'Semantle' solver this side of the Mississippi.
A simple, modern alternative to 'pycocotools'.
PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch
Build batch pipelines in Python that run anywhere.
Personal fork of microsoft/visual-chatgpt
Fastest Wordle solver in the West.
A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)