[Solved] Difference in test accuracy while running code about lbf HOT 4 CLOSED

Hi Aashish,

Thank you for pointing it out. I wrote the original codes a year ago and rewrote them after acceptance. I did not continue training after I reached ~55% accuracy using this code. I will look into it to see what's possibly wrong. Before that, you can use the performance achieved by running the code yourself.

I see your recent arxiv paper: A Weakly Supervised Model for Solving Math word Problems. Could you please cite our paper, since we are the first paper to do weakly supervised math word problem solving? You can use the accuracy that you achieved, and add a footnote suggesting that this is the accuracy by running our codes.

Best,
Yining

from lbf.

Close this issue until further update

from lbf.

Hi,
I ran the training code using "python run_seq2tree.py --model ma-fix --nstep 50 --name ma-fix". But I'm getting best test accuracy of around 53.6%. Can you please provide the exact details of the setting in which 59.4% accuracy was obtained?

Hi,

I found the bug. Just comment these two lines:

And the codes can get similar results to the paper (with probably a 1% variance).

Sorry for the inconvenience!

Again, we will really appreciate it if you could cite us in ur paper.

Best,
Yining

from lbf.

Hi, sorry to bother again. The debugged codes can get 59.8% accuracy. I have uploaded the checkpoint for it.

Best,
Yining

from lbf.