Prompt format for different models about loogle HOT 2 OPEN

Hi, we agree that dedicately customizated prompting for different tasks helps. Since it can discover the potential of models through more standardized output formats to get better performance when assessment.

As far as we know, LongBench desinged different instructions for different datasets/tasks instead of models.
In our case, we select the most popular and common NLP tasks (summarization, QA) for evaluation. There are no strict requirements on the output format of these tasks, while we indeed design prompts adaptively for cloze tasks for fair comparison.

from loogle.

Thanks for clarification!

from loogle.