ask_details pipeline evaluation process about wrenai HOT 1 CLOSED

The following is the summary of the related discussion with @cyyeh

1. The pipeline evaluation process will follow the "ask pipeline eval" methodology.
2. The initial dataset for pipeline evaluation will be the "baseball" data from the Spider dataset.
3. For creating the basic dataset needed for evaluation, we will use either GPT-3.5-turbo or GPT-4.
4. (Optional) Consider employing a Large Language Model (LLM) to rank responses and assign scores.
5. The evaluation must ensure that all queries from the evaluation steps can be executed on the core server.
6. Due to the requirement in point 5, it's also necessary to generate the CTE from the step query and verify that the results match those provided by the Spider dataset.

from wrenai.