Note:
- This project is based on Awesome Information Retrieval in the Age of Large Language Model and LLM4IR-Survey.
- Any feedback and contribution are welcome, please open an issue or contact me.
- LLM-for-IR-Rec
- REALM: Retrieval augmented language model pre-training. Kelvin Guu et.al. ICML 2020.(REALM)
- Improving language models by retrieving from trillions of tokens. Sebastian Borgeaud et.al. ICML 2022. (RETRO)
- Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study. Boxin Wang et.al. Arxiv 2023.
- Dense Passage Retrieval for open-domain question answering. Vladimir Karpukhin et.al. EMNLP 2020. (DPR)
- RAG: Retrieval-augmented generation for knowledge-intensive NLP tasks. Patrick Lewis et.al. NeurIPS 2020. (RAG)
- FiD: Leveraging passage retrieval with generative models for open domain question answering. Gautier Izacard, Edouard Grave EACL 2021.
- Distilling Knowledge from Reader to Retriever for Question Answering. Gautier Izacard et.al. ICLR 2021 (FiD-KD)
- Copy Is All You Need. Tian Lan et.al. ICLR 2023. (COG)
- Active Retrieval Augmented Generation. Zhengbao Jiang et.al. Arxiv 2023. (FLARE)
- Generalization through memorization: Nearest neighbor language models. Urvashi Khandelwal et.al. ICLR 2020. (kNN-LM)
- Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. Harsh Trivedi et.al. ACL 2023. (IRCoT QA)
- Rethinking with retrieval: Faithful large language model inference. Hangfeng He et.al. Arxiv 2023.
- Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation Ruiyang Ren et.al. Arxiv 2023.
- Retrieve Anything To Augment Large Language Models. Peitian Zhang et.al. Arxiv 2023. (LLM-Embedder, supports the diverse retrieval augmentation needs of LLMs with one unified embedding model)
- Atlas: Few-shot Learning with Retrieval Augmented Language Models. Gautier Izacard et.al. Arxiv 2022.
- REPLUG: Retrieval-Augmented Black-Box Language Models. Weijia Shi et.al. Arxiv 2023.(REPLUG)
- Learning to Retrieve In-Context Examples for Large Language Models. Liang Wang et.al. Arxiv 2023.
- InPars: Data augmentation for information retrieval using large language models. Luiz Bonifacio et.al. SIGIR 2022. (InPars)
- UPR: Improving passage retrieval with zero-shot question generation. Devendra Singh Sachan et.al. EMNLP 2022. (UPR, point-wise, query generation)
- Promptagator: Fewshot dense retrieval from 8 examples. Zhuyun Dai et.al. ICLR 2023. (Promptagator)
- Precise Zero-Shot Dense Retrieval without Relevance Labels. Luyu Gao et.al. Arxiv 2022. (HyDE)
- Text and Code Embeddings by Contrastive Pre-Training, Neelakantan et al., arXiv 2022.
- Large Dual Encoders Are Generalizable Retrievers, Ni et al., ACL 2022.
- Task-aware Retrieval with Instructions, Asai et al., ACL 2023 (Findings).
- Transformer memory as a differentiable search index. Tay et al., NeurIPS 2022.
- Large dual encoders are generalizable retrievers. Jianmo Ni et.al. emnlp 2022. google(GTR, Generalizable T5-based dense Retrievers)
- Vector Search with OpenAI Embeddings: Lucene Is All You Need. Jimmy Lin et al. arXiv 2023. (OpenAI’s ada2 embedding+HNSW(Lucene search library), rather than Faiss library)
- Language Models are Universal Embedders. Xin Zhang et.al. arXiv 2023. (Udever,across tasks, natural and programming languages)
- Fine-Tuning LLaMA for Multi-Stage Text Retrieval. Xueguang Ma et.al. arXiv 2023. (RepLLaMA, RankLLaMA, fine-tune LLaMA-2 with bi-encoder architecture)
- Making Large Language Models A Better Foundation For Dense Retrieval Chaofan Li et.al. arXiv 2023. (pre-train llama + RepLLaMA)
- Improving Text Embeddings with Large Language Models. Liang Wang et.al. arXiv 2024. (E5mistral-7b, use gpt-4 to build multi-task multi-language synthetic data (<q,d+,d->) to fine-tune Mistral-7b)
- Sgpt: Gpt sentence embeddings for semantic search. Muennighoff. arXiv 2022. (SGPT)
- Scaling Sentence Embeddings with Large Language Models Jiang et.al. arXiv 2023. (PromptEOL, instruct the model to summarize the input in a single word and then take the last token embedding + In-context Learning Example(zero-shot) / QLora with Contrastive Learning Objective(Fine-tuning))
- One Embedder, Any Task: Instruction-Finetuned Text Embeddings Su et.al. arXiv 2023. (INSTRUCTOR)
- GRIT:Generative Representational Instruction Tuning Muennighoff et. al. arXiv 2024. (GRIT unifies representational instruction tuning and generative instruction tuning)
- Repetition Improves Language Model Embeddings Springer et. al. arXiv 2024. (echo embeddings, repeat the input twice in context and extract embeddings from the second occurrence)
- Generate rather than retrieve: Large language models are strong context generators. Wenhao Yu et.al. ICLR 2023. (Retrieve-then-Read->Generate-then-Read pipeline)
- Large Language Models are Built-in Autoregressive Search Engines, Ziems et al., ACL 2023 (Findings). (LLM-URL, Given query, llm directly generate Web URLs, open domain QA)
- Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts. Arian Askari et.al. Arxiv 2023. (ChatGPT-RetrievalQA dataset)
- Document Ranking with a Pretrained Sequence-to-Sequence Model, Nogueira et al., EMNLP 2020 (Findings).
- Text-to-Text Multi-view Learning for Passage Re-ranking, Ju et al., SIGIR 2021 (Short Paper).
- The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models, Pradeep et al., arXiv 2021.
- RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses, Zhuang et al., SIGIR 2023 (fine-tune T5 with ranking losses).
-
Improving Passage Retrieval with Zero-Shot Question Generation, Sachan et al., EMNLP 2022.
-
Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-ranker, Cho et al., ACL 2023 (Findings).
-
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agent. Weiwei Sun et.al. Arxiv 2023. (RankGPT, list-wise, sliding window)
-
Zero-Shot Listwise Document Reranking with a Large Language Model. Xueguang Ma et.al. Arxiv 2023. (LRL, list-wise, sliding window)
-
Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. Qin et al. arXiv 2023. (pairwise)
-
Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels. Honglei Zhuang et.al. arXiv 2023.
-
GAR-meets-RAG paradigm for zero-shot information retrieval. Daman Arora et.al. Microsoft. arXiv 2023. (RRR: Rewrite-Retrieve-Rerank, use RAG model to provide the context for the GAR model, use GAR model as the retriever model for the RAG model.)
- Query Understanding in the Age of Large Language Models. Avishek Anand et.al. Gen-IR 2023.
- Generative relevance feedback with large language models. Iain Mackie et.al. Arxiv 2023.
- Query2doc: Query expansion with large language models. Liang Wang et.al. Arxiv 2023.
- Generate, Filter, and Fuse: Query Expansion via Multi-Step Keyword Generation for Zero-Shot Neural Rankers. Minghan Li et.al. arXiv 2023. (GFF, a query expansion pipeline to imporve zero-shot ranker)
- Large Language Model based Long-tail Query Rewriting in Taobao Search. Wenjun Peng et.al. Arxiv 2023.
- IterCQR: Iterative Conversational Query Reformulation without Human Supervision. Yunah Jang et. al. Arxiv 2023. (reformulating conversational queries into stand-alone questions, which enables the utility of off-the-shelf retrievers)
- LLMs may Dominate Information Access: Neural Retrievers are Biased Towards LLM-Generated Texts. Sunhao Dai et.al. Arxiv 2023.
LLM for IR
- Perspectives on Large Language Models for Relevance Judgment Guglielmo Faggioli et.al. ICTIR 2023. (Best paper)
- Large Language Models for Information Retrieval: A Survey Yutao Zhu et.al. Arxiv 2023.
Retrieval Augmented LLM
- Retrieval-based Language Models and Applications Akari Asai et.al. ACL 2023. (Tutorial)
- Augmented Language Models: a Survey Grégoire Mialon et.al. Arxiv 2023.
- Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community Qingyao Ai et.al. Arxiv 2023.
- KILT: a benchmark for knowledge intensive language tasks. Fabio Petroni et.al. NAACL 2021.
-
Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5). Shijie Geng et.al. RecSys 2022.
-
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems. Zeyu Cui et.al. arXiv 2022.
-
Prompt Distillation for Efficient LLM-based Recommendation . Lei Li et.al. CIKM 2023
-
Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach. Junjie Zhang et.al. arXiv 2023.
-
Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. Yunfan Gao et.al. arXiv 2023.
-
TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. Keqin Bao et.al. RecSys 2023.
-
LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking. Zhenrui Yue et.al. arXiv 2023.(LLM ranker, map the item to the corresponding index letter)
- A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systems. Keqin Bao et.al. arXiv 2023.
- CoLLM: Integrating Collaborative Embeddings into Large Language Models for Recommendation. Yang Zhang et.al. arXiv 2023.
- Large Language Model Can Interpret Latent Space of Sequential Recommender. Zhengyi Yang et.al. arXiv 2023.
-
Large Language Models for Generative Recommendation: A Survey and Visionary Discussions. Lei Li et.al. arXiv 2023.
-
When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities. Jin Chen et.al. arXiv 2023.
-
How Can Recommender Systems Benefit from Large Language Models: A Survey. Jianghao Lin et.al. arXiv 2023.
-
Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems. Peng Liu et.al. arXiv 2023.
-
A Survey on Multi-Behavior Sequential Recommendation. Xiaoqing Chen et.al. arXiv 2023.
-
Robust Recommender System: A Survey and Future Directions. Kaike Zhang et.al. arXiv 2023.
-
Is ChatGPT a Good Recommender? A Preliminary Study. Junling Liu et.al. CIKM 2023 GenRec Workshop.
- Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation. Jizhi Zhang et.al. RecSys 2023