REINFORCE PolicyEstimator loss about reinforcement-learning HOT 4 CLOSED

1. Yes, your understanding seems right. If you want to know why this works in detail I recommend watching the lecture or reading the book, they have goof explanations. Intuitively, you update the policy function towards actions that give a better reward.

In English, what you do is adding up the discounted actual rewards for each state in the episode given the future states of that state, correct?

Yes.

And this applies even if a state is visited more than once during the episode. In this case the same state can be updated more than once?

Yes. I don't think there's a reason why a state shouldn't be updated more than once.

think you want to say that we can use Q-Learning OR SARSA in this example (not both), correct?

I removed the comments from the notebook because they may be confusing. But yes, you can use either one, not both.

I wonder if you will implement eligibility traces soon or at least give a hint how to implement in the simplest way.

I will try to add that, but implementing some of the missing algorithms like A3C is probably higher priority.

from reinforcement-learning.

Thank you

from reinforcement-learning.

As far as I understand, the main difference between Actor Critic and A3C is that A3C is using multiple independent agents instead of one agent.

In other words:

A3C = Actor Critic + Some tricks such as Asynchronous multiple agent parameter updates.

correct?

from reinforcement-learning.

Yes.

from reinforcement-learning.