Why append additional (s a r) pair to the replay buffer after one episode is done? about pytorch-ddpg HOT 3 OPEN

I think this is weird, too.

agent.memory.append(
                observation,
                agent.select_action(observation),
                0., False
            )

Also, done is set to False is this tuple, which is more perplexing.

from pytorch-ddpg.

Having said so, I think this would probably have a negligible effect in terms of learning, given that the replay buffer is so big, but I think it's good for the author to check on this @ghliu .

from pytorch-ddpg.

In Buffer's code，I guess the terminal state can be used to divide transitions from each episodes，in terms this，I think maybe it‘s a bug.

from pytorch-ddpg.