This is the implementation of SpaceByte used in SpaceByte: Towards Deleting Tokenization from Large Language Modeling.
SpaceByte is a tokenization-free large language model (LLM) that uses multiscale modeling at the byte and word levels to model language with the same performance as standard LLM architectures that instead use tokenization.
See the reproduce
directory for instructions for reproducing the results in our paper.
lessw2020 / spacebyte Goto Github PK
View Code? Open in Web Editor NEWThis project forked from kjslag/spacebyte
A byte-level decoder architecture that matches the performance of tokenized Transformers.