SARSA and Value Function Approximation about reinforcement-learning HOT 1 CLOSED

I think you want to say that: it is ok to use SARSA instead of Q-Learning in this specific example.

I left the code there to demonstrate the SARSA and Q-Learning is almost the same, except for a small change in the code.

However, In case of non linear approximator (neural network as in DQN), we usually use reply memory to break the correlation. Using reply memory will limit us with off-policy algorithms (such as Q-learning), and we can not/should not use on-policy algorithms (such as SARSA) while using reply-memory. (I believe the reason is that: When using reply memory, the sampled transitions taken from the reply-memory are actually generated using old-version of the policy not the current version)

Yes, I believe this is correct. You cannot use replay memory with on-policy algorithms. However, you can use asynchronous mehtods (like A3C) that break correlations with on-policy algorithms.

then the commented code may work, only because in this specific code, you are using a simple linear approximator and not using reply memory, correct?

Yes, SARSA still works because I am not using a replay memory.

from reinforcement-learning.