covariance matrix about ppo-for-beginners HOT 1 CLOSED

The covariance matrix is used to determine the shape of our multivariate gaussian distribution around some mean (which is generated by the actor network); this distribution gives us a way to explore values around our mean. The actor network is not trying to learn the covariance matrix, but rather, it's trying to learn the general vicinity an action should be sampled from, given a state, such that the future expected return is maximized.

from ppo-for-beginners.