dialevalmetrics's People
Forkers
chateval tensixchuan shubhamphal soundarya98 y12uc231 srt2157 fabriceyhc maxlchen trellixvulnteam lainwiredin adamlin120dialevalmetrics's Issues
regr.pkl file not available
when i run "python usr_fed/usr/usr_server.py", error arrears:
Traceback (most recent call last):
File "usr_server.py", line 1, in
import usr
File "/home/zper/DialEvalMetrics/usr_fed/usr/usr.py", line 6, in
import regression
File "/home/zper/DialEvalMetrics/usr_fed/usr/regression.py", line 6, in
with open('regr.pkl','rb')as f:
FileNotFoundError: [Errno 2] No such file or directory: 'regr.pkl'
after i wget https://github.com/Shikib/usr/blob/master/examples/regr.pkl from original usr model github,
error also appears:
pickle invalid load key, '\x0a'.
i wonder if you can provide your pkl file, thanks.
如何获取bert-ruber分数和PONE分数
你好,请问bert-ruber和PONE有什么区别呀
我发现使用gen_data.py创建bert-ruber的数据时,产生的数据是放在了pone文件夹中,运行相应run之后,会产生三个文件:refer_score_dailydialog_sampled.json、ruber_score_dailydialog_sampled.json、unrefer_score_dailydialog_sampled.json,我想问一下这三个文件分别代表什么?
哪一个是bert-ruber的分数,哪个是pone的分数呀?
能不能单独使用其中一个评价指标呀?
比如我想单独直接使用ADEM,运行ADEM里面的eval.py
Unable to find "run_eval.sh"
Modify the path in run_eval.sh as specified in the script, since we need to activate Conda environment when running the script. Run eval_metrics.sh to evaluate all quality-anntoated data.
I can't find "run_eval.sh". Who knows where it is?
Unable to reproduce results
I'm unable to get the desired results by following the steps as given in the README due to multiple issues such as:-
- Environment creation failures for some of the them
- gen_data.py is giving errors for DSTC6 and Engage
- Transformers version issues for FBD, Maude (Eval_base environment) and TF version issues for run_bert_as_service.sh
and some others as well. - Various files such as restore/ensemble.yml, ./data/DailyDialog/keyword.vocab etc. not being present
So I request to update the repository with the latest codes and files if possible to help get the results as highlighted in your paper.
Unable to reexperiment with GRADE
So thanks for your existing work !
I have get the result of grade_convai and grade_dailydialog, for both pearson and spearman scores are the same with yours. While for grade_empatheticdialogues, I get the different results under the same computation and evaluate methods.
- First I use nlg_eval to compute baseline metrics for both three datasets
- I modify the human_correlation.py script in data/grade_data, to compute for all the scores in both three datasets, and as described in above, two of them get the same but last one is different.
Waiting for your suggestions, thanks !
Turn-level DSTC9 annotation data
I noticed there is just dialog-level DSTC9 data used in your work. May I ask if turn-level DSTC9 annotation data is made public? How can I get this data? Thank you very much!
conda install:Found conflicts! Looking for incompatible packages.
when i run "conda env update --name eval_base --file conda_envs/eval_base/environment.yml" on mac,
error reports :
Collecting package metadata (repodata.json): done
Solving environment:
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
Solving environment:
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with a past
explicit spec that is not an explicit spec in this operation (libgcc-ng):
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.