Add log probabilities about ai-toolbox HOT 6 CLOSED

That hasn't really been on my radar yet.

I guess using just log probabilities in the transition functions won't really be useful unless all algorithms are also modified to take advantage of that, and that would require modifying the entire library.

What changes are you thinking of in particular?

from ai-toolbox.

I agree. Looking at the original VI algorithm, for example, and looking at other MDP implementations, it doesn't seem like using log probabilities is really a thing. Makes me think there's maybe a reason for that...

I'm working with the MDP library with a biggish state-space and small probabilities, so I'm curious to see if using logs will clear the small summing errors I'm getting. I'll try and modify the algorithms and see if it makes any difference and make a pull request if you want to take a look

from ai-toolbox.

I'm not 100% sure; not using logs definitely helps in that I can use matrix operations from Eigen untouched in order to make computations rather than rolling my own - I'm not sure whether a direct log equivalent exists.

Feel free to try it out and let me know if anything comes out of it. The idea for this library is that it should be not too hard to customize, so in case you have any problems understanding/touching some pieces feel free to open issues so that I can document stuff better!

from ai-toolbox.

I think the problem comes with the fact that ln(0) = -inf. How do you use that in practice?

from ai-toolbox.

Fair point. Would there be significant algorithmic/mathematical implications if you ignore cases where the transition probability is zero? My logic being that if an event can't occur (e.g. transition from s -> s1 with action a), do we need to represent that or even worry about it at all? I agree that small probabilities (<10e-6 or whatever) can add unwanted noise though

from ai-toolbox.

Would there be significant algorithmic/mathematical implications if you ignore cases where the transition probability is zero?

I don't think so.

do we need to represent that or even worry about it at all?

Not really. The problem is that what you propose is some kind of forbidden states/actions system. AFAIK it's not very well supported by traditional solving methods that need to have a stable set of actions or states (all of them really), hence the probability at 0 to emulate that.
Everything can be modified though but it might be a bit of work.

from ai-toolbox.