This is the official repository for the paper "From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty." Our research investigates undesirable behaviors in large language models (LLMs), such as hallucinations and sequence repetitions, identifying them as fallback mechanisms under uncertainty. We found that as models become more advanced, they shift towards more complex fallback behaviors.

For details about our experiments and findings using Llama 2 and 3, OLMo, and Pythia models across various settings, please see our paper.

🚨 All details about the code, datasets, and instructions on how to reproduce our experiments will be updated soon.

We will upload the setup instructions soon.

The data for the different experiments is available in the data directory. It will be uploaded to HuggingFace hub as well soon.

We will provide instructions on how to generate and analyze outputs from the models soon.

We will provide instructions on how to create the plots soon.

If you find our work useful, please cite our paper as follows:

@article{ivgi2024fallbacks,
  title={From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty}, 
  author={Maor Ivgi and Ori Yoran and Jonathan Berant and Mor Geva},
  year={2024},
  journal={arXiv:2407.06071},
  url={https://arxiv.org/abs/2407.06071}, 
}