Comments (6)
Hi,
The current implementation of ROME is based on the original paper. It's important to retain this baseline, so I will reintroduce r-ROME as a new method in EasyEdit, currently in development.
Regarding your second question, it should be rephrase_prompts instead of rephrased_prompts. The parameter you passed is ignored because it's not recognized. You can try modifying this and see if it resolves the issue.
from easyedit.
Hi,
Thank you, it worked.
Two more questions.
Does sequential_edit=True makes the model be evaluated after all the edits?
Where can I find the specific information on how default portability and locality are computed?
Thank you very much for your time and input :)
from easyedit.
And maybe another one.
In IKE having:
train_ds = [
{
'prompt': 'Q: The president of the US is? A:',
'target_new': 'Joe Biden',
'rephrase_prompt': 'The leader of the United State is',
'locality_prompt': 'The president of Russia is ',
'locality_ground_truth': 'Putin'
},
{
'prompt': 'Einstein specialized in',
'target_new': 'physics',
'rephrase_prompt': 'Einstein is good at',
'locality_prompt': 'Q: Which subject did Newton specialize in? A: ',
'locality_ground_truth': 'physics'
},
# add more if needed
]
what syntax should i use to add multiple locality and portability prompts?
from easyedit.
Hi,
Thank you, it worked.
Two more questions.
Does sequential_edit=True makes the model be evaluated after all the edits? Where can I find the specific information on how default portability and locality are computed?
Thank you very much for your time and input :)
Q1: Yes
Q2: You can get the corresponding metrics by entering locality_inputs
and portability_inputs
in the format (https://github.com/zjunlp/EasyEdit?tab=readme-ov-file#baseeditor)
from easyedit.
The data format for both is a dict, for each measurement dimension, you need to provide the corresponding prompt and its corresponding ground truth.And maybe another one.
In IKE having:
train_ds = [ { 'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'rephrase_prompt': 'The leader of the United State is', 'locality_prompt': 'The president of Russia is ', 'locality_ground_truth': 'Putin' }, { 'prompt': 'Einstein specialized in', 'target_new': 'physics', 'rephrase_prompt': 'Einstein is good at', 'locality_prompt': 'Q: Which subject did Newton specialize in? A: ', 'locality_ground_truth': 'physics' }, # add more if needed ]
what syntax should i use to add multiple locality and portability prompts?
from easyedit.
Thank you for you massive help, got everything!
from easyedit.
Related Issues (20)
- 使用mend预训练llama时出现的报错 HOT 2
- 数据集问题 HOT 2
- counterfact数据结果 HOT 11
- ROME Question HOT 8
- [Speed issue]: How would you recommend running EasyEdit faster HOT 3
- [Evaluation Issues]: T5 Results are really strange HOT 3
- Evaluation Question HOT 4
- locality and portability evaluation HOT 8
- GRACE sequential edit result HOT 15
- IKE fluency HOT 2
- ccks gpt2-xl 模型 为什么用roberta模型加载? HOT 2
- R-ROME has poor performance when using GPT2-xl HOT 2
- WISE CONTEXT_TEMPLATES_CACHE HOT 7
- WISE tokenize HOT 7
- what is the meaning of archive HOT 2
- [Wise] Editing loss = nan + 0 HOT 2
- AttributeError: Can't pickle local object 'length_collation.<locals>.collate_fn' HOT 8
- Issue with KN batch editing HOT 2
- Question about ZSRE experiment, how to reproduce results in your WISE paper? HOT 6
- PermissionError: [Errno 13] Permission denied: './results/models/MEND/gpt2-xl' HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from easyedit.