Towards reliable, controllable and more efficient generation with Large Language Models (LLMs)
A curated list of papers related to constrained decoding of LLM, along with their relevant code and resources.
Library | Feature | Stars |
---|---|---|
guidance-ai/guidance | CFG, Regex, JSON Schema, Token Forcing, compatible with Transformers, LLAMA-CPP | |
outlines-dev/outlines | CFG, Unicode support, Hugging Face ecosystem, VLLM support | |
sgl-project/sglang | Regex support, emphasis on LLM inference efficiency, compressed FSM | |
eth-sri/lmql | Regex support, various constraints, more powerful control flow | |
jxnl/instructor | Try-Reject-Repeat approach to ensure constraints are met | |
noamgat/lm-format-enforcer | Regex, JSON Schema, Beam Search etc. | |
epfl-dlab/transformers-CFG | CFG (EBNF Interface), Compatible with Transformers, Easy to extend for research | |
uiuc-focal-lab/syncode | CFG generation that supports builtin grammars like JSON, Python, Go, and more |
Disclaimer:
- The libraries listed above are not exhaustive and are subject to change.
- The features mentioned are 100% not exhaustive and I strongly recommend checking the respective repositories for more details.
- The libraries are listed by the Github stars
- If you are the author of a library and would like to add or update the information, please open an issue or submit a pull request.
Papers with are newly added papers (not necessarily newly published papers).
Date | Paper | Publication |
---|---|---|
2024-05 | COLLIE: Systematic Construction of Constrained Text Generation Tasks | ICLR |
2023-12 | BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing | NeurIPS Track on Datasets and Benchmarks |
2023-10 | Evaluating Large Language Models on Controlled Generation Tasks | Arxiv |
2023-09 | Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? | Arxiv |
2020-12 | CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning | EMNLP Findings |
Date | Paper | Publication |
---|---|---|
2024-04 | "We Need Structured Output": Towards User-centered Constraints on Large Language Model Output | Arxiv |
- Leveraging Constrained Sampling for Fill-in-the-Middle Code Completion by nielstron
- Proper Well-Formedness for Finite LLM Sampling by nielstron
- LLM Decoding with Regex Constraints by Vivien
- Constrained Decoding is Posterior Inference by Saibo-creator
- Making Structured Generation Faster Than Unstructured
- Coding For Structured Generation with LLMs
- Beating GPT-4 with Open Source
- Prompt Efficiency - Using Structured Generation to get 8-shot performance from 1-shot.
- How fast can grammar-structured generation be?
- Structured Generation Improves LLM performance: GSM8K Benchmark
- Coalescence: making LLM inference 5x faster
Many of the blogs are written by Outlines team, many thanks to them for their great work! ❤️
This list is not exhaustive and will be updated regularly. If you have any suggestions or want to add a paper, please feel free to open an issue or submit a pull request. We hope to include all relevant papers in this list.
Contributions are welcome! Feel free to submit a pull request or open an issue. Please make sure to read the Contributing Guidelines before contributing.