aicrumb / blog Goto Github PK

Support development (please):
btc on bitc network: 3JB6if8iTpWBbBGBdnoYZxpg3CZoLUUvYe
eth on eth unified 0x3fad449e20b44308d80ec663cbc2408f8bb68804
sol on sol EpxKy2DqsoALJC35BJ3ZPEoqPYqwQmVGC1BpqeT6vJHt
xtz tz1ULvqesQA8SnopRzuQKFQj2jdLGBatJoC3

HuggingFace - https://huggingface.co/crumb

Jan 2024 - https://aicrumb.github.io/reports/012024/

• GLORT2 (Glort2 Low Rank Transformer-Transformer), a transformer model where every linear layer is replaced with a smaller transformer.
• https://huggingface.co/crumb/GLORT2

• Creating very small continued-pretraining datasets specifically to increase the size of the domain that a model performs well in
• https://crumbly.medium.com/reweighting-refining-ai-with-precision-and-efficiency-9a01c2ca4c26

• Mixture-of-experts LoRA, the first of it's kind on the Llama-2 models! (and looks like one of the first few finetunes!)
• medium blog post with proof of concept and inference code: https://crumbly.medium.com/llama-2-molora-f5f909434711

• Knowledge distillation in Pythia models
• Model card (which is also the project page) https://hf.co/crumb/distilpythia

• novel method for estimating single-token representations (text-inversions) of images quickly
• https://twitter.com/aicrumb/status/1588769725286600704?s=20&t=DW43TNWTqs-lYp63H3CqGg

• Investigated the effectiveness of certain adversarial attacks on AI tools
• https://twitter.com/aicrumb/status/1580440734192148481 (explainer thread)
• https://colab.research.google.com/github/aicrumb/notebook-hosting/blob/main/Adversarial_Attacks_to_AI_Art_Tools.ipynb (messy code)

• Tuned a prompt-generating BLOOM model for aesthetic Stable Diffusion results
• One of the first open source rlhf'd language models I guess (possibly the first one on huggingface)? Sucks that that achievement is on a stable diffusion prompt generator but whatever.
• https://twitter.com/aicrumb/status/1598248104981319681

• Early CLIP Guided Text2img notebook built on top of Stable Diffusion
• https://hf.co/doohickey