Hi Jiachen,

Sorry for disturbing. In your paper/Appendix B-Algorithms/Algorithm 2, there's V(pi_n+1; w)-V(pi_n; w) which corresponds your below code: # TD error = r + gamma * v(s'; w) - v(s; w); delta = reward + discount*(vec_features_next.dot(ac.w)) - (vec_features.dot(ac.w))
May I ask what's the math definition of Value function? I don't find the specific math definition of the Value function, but only see the definition of Value function in your code as discount*(vec_features_next.dot(ac.w)). Is there some reference or math formula can help me to understand V(s, w)=discount*(vec_features_next.dot(ac.w))?
Thank you very much in advance!

Hi Jiachen, sorry for disturbing. It seems that the log likelihood log(F) has multiple local maxima in terms of theta.
Below test results show one local maxima between theta=7 and 8 and others between theta=9 and 10, 12 and 13, 15 and 16, 17 and 18. Is there some way to determine the global maxima of log(F)(theta)? Thank you very much in advance!

Hi,

Could you provide some sample data (train_round2, etc)? I am not sure what the expected format is of the input data, but an example input would be incredibly helpful to modify my data to work with the infrastructure in place.