Can not reach groundhog performance about blocks-examples HOT 7 CLOSED

We were getting comparable scores for cs-en when the initial pr was made, around august so the issues in NMT repo might be outdated. iirc the there were fixes at beam-search which uses the generate computational graph (same one we generate samples).

Have you checked whether the cost computational graphs are generating the same cost or not (using the same batch and initial parameters)?

from blocks-examples.

Thanks for your fast response,
we didn't try that yet. It's next on the list of things to try. Right now we are looking into something else, I let you know if we find something.

from blocks-examples.

Thanks, keep us posted

from blocks-examples.

Henry Choi told me that he was able to reproduce English to French results
with this implementation.

On 8 January 2016 at 15:32, Orhan Firat [email protected] wrote:

Thanks, keep us posted

—
Reply to this email directly or view it on GitHub
#71 (comment)
.

from blocks-examples.

@critias Hi, I am wondering did you reach the Groudhog performance. If you did, how did you reach that? I am trying the example as well and I cannot reach the performance.

from blocks-examples.

Hi,
yes and no. We got roughly equal results on the validation set during training, but not after reloading the saved model. Since we changed the code base a little to reload and translate the model I guess the error is on our side. It's still kinda unclear and we have to look into this in more detail, but were busy with other things last week.
Beside that we also try using orhanfs fork to see if his code to translate works better for us.

from blocks-examples.

It turned out the problem was on our side. We changed some minor parts of the code that caused a mismatch between the encoding used to create the vocabulary (just bytes) and the encoding used during training/translation (unicode).
We are now able to reproduce the GroundHog results and even slightly surpassed it (0.4% Bleu).
I'll close the issue. Thanks for your help and keep up the good work.

from blocks-examples.