johnpion-zhang / llama2-webui Goto Github PK
View Code? Open in Web Editor NEWThis project forked from liltom-eth/llama2-webui
Run Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Supporting Llama-2-7B/13B/70B with 8-bit, 4-bit. Supporting GPU inference (6 GB VRAM) and CPU inference.
License: MIT License