benkrause / dynamic-evaluation Goto Github PK
View Code? Open in Web Editor NEWDynamic evaluation for pytorch language models, now includes hyperparameter tuning
License: BSD 2-Clause "Simplified" License
dynamic-evaluation's People
Stargazers
Watchers
Forkers
vikingmew walidar codeaudit wuzeen strategist922 shubhampachori12110095 xsongx lim0606 jemxgw timeflow-lab hainan-xv esvhd sunnydreamrain ksferguson timtai1 digifaire srgangireddy silverthorne25 sn-donbenjamindynamic-evaluation's Issues
Updates for pytorch 0.4
Hi there,
Has anyone tried to update this code for pytorch 0.4? awd-lstm-lm repo recently upgraded to make the models work in 0.4.
I made some attempts but got stuck at gradstat(), since in this method the code calls model.eval(), I encountered the following error while running with a trained model. See code here
Traceback (most recent call last):
File "dynamiceval.py", line 299, in <module>
gradstat(args, corpus, model, train_data, criterion)
File "dynamiceval.py", line 83, in gradstat
loss.backward()
File "/usr/anaconda3/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/usr/anaconda3/lib/python3.6/site-packages/torch/autograd/__init__.py", line 89, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: backward_input can only be called in training mode
Looks like I cannot call loss.backward() when the model is set to eval mode. I've not tried running in with previous versions of pytorch. Is there a workaround for this in version 0.4?
Thanks.
Reproduce PTB results
Hi,
Thanks for sharing this work.
I tried to reproduce the PTB results (the first two bold rows in Table 1):
| architecture | val | test |
|---|---|---|
| LSTM(ours) | 88.0 | 85.0 |
| LSTM+dynamic eval | 73.5 | 71.7 |
However, I am getting results:
python main.py --cuda --tied #valid ppl 90.69 and test ppl 87.08 python dynamiceval.py --model model.pt --val #perplexity of 79.7848 python dynamiceval.py --model model.pt #perplexity of 78.9188
where main.py (the baseline) is from https://github.com/pytorch/examples/tree/master/word_language_model, and
dynamiceval.py is from this repository.
There is over 13 ppl difference (from 85.0 to 71.7) on the test data reported in the paper. However, my results show that there is 7.3 ppl difference (from 87.08 to 79.7848).
Did I do anything wrong?
Thanks,
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.