Using the Toolbox to solve the Tag problem about ai-toolbox HOT 4 CLOSED

1

I'm not sure whether you noticed that every function is documented. If in the Python command line you type

help(AIToolbox.POMDP.PBVI)

you'll get a full explanation of the class and methods. The docs are also pretty much the same as the equivalent C++ version, so feel free to go over there as well.

Defining a VFun in Python is possible, but in general these types are symmetric to the C++ ones (when possible, sometimes it is not possible to build them in Python directly, depending on what I have wrapped). It's mostly a list of lists. The documentation for the lower level stuff exists, but might be a bit more hidden. In general I would advise looking around a bit in the C++ header files (in the include folder) to orient yourself even if you do not plan to use C++.

In any case, the docs certainly could use improvement, so if you feel something is unclear and have ideas to make them better, please do open an issue and we can discuss how you can contribute :)

2

The toolbox at the moment does not include changing action spaces. This was a design decision that I made at the beginning, as with action spaces that depend on the current state every single algorithm needs to be aware of that and have code which handles lists of actions for each state.

You can emulate that by simply having the north action do something else. This might make learning/planning a bit more inefficient since you have dupes, but trust me when I say that it greatly simplifies the code of the library. Since I'm mostly maintaining this alone, I figured it would be a reasonable tradeoff.

from ai-toolbox.

right that make sense!

so is it appropriate that every time i use PBVI, i just pass an empty VFun to it?

and one more question is that how will the toolbox deal with it if i want the markov process to stop when hitting certain state?

from ai-toolbox.

The default parameter should be empty anyway, so I think you can also not pass anything and it should work the same.

For planning algorithms, I usually just have my MDP return the same state over and over with 0 reward. For learning, you just stop the episode when you reach the terminal state.

from ai-toolbox.

alright! Problems solved! you are really helpful! :)

from ai-toolbox.