YiVal
Introduction
YiVal is an innovative open-source project, rooted in the principles of the Yijing, focused on revolutionizing the end-to-end evaluation process of AI models. We aim to provide comprehensive insights into not only the individual components such as models and prompts but more importantly, the final output produced by their interaction.
Using Yival for auto prompt generation
Roadmap
Qian (The Creative, Heaven) 🌤️ (乾):
- Setup the framework for wrappers that can be used directly
in the production code.
- Set up the BaseWrapper
- Set up the StringWrapper
- Setup the config framework
- Setup the experiment main function
- Setup the evaluator framework to do evaluations
- One auto-evaluator
- Ground truth matching
- Human evaluator
- Interactive evaluator
- Reader framework that be able to process different data
- One reader from csv
- Output parser - Capture detailed information
- Documents
- Git setup
- Cotribution guide
- End2End Examples
- Release