Neural Net Input format about untouchablethunder HOT 2 CLOSED

Yeah, you've read it correctly. We have stacked maps for each object type and the mathematical object this becomes is a tensor rather than a matrix. One thing to note is that some of the maps are abstract types (WLOG e.g. slice 4 of the tensor).

It would be simple to use the pictures instead of the tile-maps, but doing so would require the conv-net to spend a bunch of time learning pixel-associations (which is standard in RL-literature, but it's part of the reason that RL systems take a long time to train using images). But since PINSKY is already such a computationally expensive system AND since the PINSKY agent-env pairs are continually changing that would change the learning problem too often (remember we're cycling in new environments frequently). Therefore, to get the system to run, we thought it would be better to remove the networks needing to learn how to parse pictures and provide an input setting that was scalable and generic for each game.

Also, when you use pictures (even tile-blocked version, most likely) you often need fancier RL techniques such as experience replay buffers.

That's an interesting thought though that would be worth exploring if the system were to lean more into the RL side (which is built-in already). I haven't tried using color-blocks. In this paper, we block out the avatar with a single-color block, and that helps with generalization of RL agents (when combined with other data-augmentation techniques).

from untouchablethunder.

@aadharna Thanks for answer!

from untouchablethunder.