Comments (4)
According to the LIMA paper, the evaluation loss is not correlated with the final human evaluation. Thus, in the context of LLM fine-tuning, "overfitting" may either improve or degrade performance.
from openchat.
According to the LIMA paper, the evaluation loss is not correlated with the final human evaluation. Thus, in the context of LLM fine-tuning, "overfitting" may either improve or degrade performance.
Oh, thanks for your information.
Can I ask if your group just take the last checkpoint for inference, or also use ChatGPT to score the effect of each checkpoint and decide which one to use for inference?
from openchat.
We used the last checkpoint for inference. We have run GPT-4 scoring on different sets of training epochs/hyperparameters etc. to select the best one.
from openchat.
Closing this issue now. If you have further questions, plz re-open.
from openchat.
Related Issues (20)
- Can you please update requirements.txt file with versions HOT 2
- Does '~80K cleaned ShareGPT data' refer to ' sharegpt_clean.json' files in the 'openchat/openchat_sharegpt4_dataset'? HOT 1
- How to user llama_convert_and_add_eot_token.py script? HOT 1
- Couldn't inference with gpu HOT 2
- AssertionError: pydantic.dataclasses.dataclass only supports init=False HOT 2
- Detailed Training setting HOT 1
- Can not reproduce the alpaca_eval results of openchat v1 HOT 5
- Not able to run openchat v1 through huggingface, not able to make correct use of conversation templates HOT 1
- 能不能出一个中文说明? HOT 1
- What's the reason for deleting the llama_convert_and_add_eot_token script? HOT 4
- Which flash attention version is being used? HOT 1
- CUDA out of memory on 8xA100 GPUs HOT 10
- Can I use system prompt when training? HOT 2
- Adding conversion scripts for open llama models
- conversation issue
- Does the training code support open_llama_3b_v2 HOT 6
- Online demo down? HOT 3
- Do I need to handle the chat history when using the curl example? HOT 1
- Installation is a mess, instructions are a mess HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openchat.