ritwikbera / podnet Goto Github PK

Currently, options are represented with a two-dimensional array but one should suffice. For example, change option representation from [[1,0,0],[1,0,0]] to [1,0,0]. Gumbel softmax and everything else still applies.

https://alexandervandekleut.github.io/gym-wrappers/

Add rnn pack_pad_sequences functionality to help PODNet process variable length trajectories in the same minibatch

Parallelize sequence encoder of PODNet with a Transformer style architecture. Create model.

https://github.com/ritwikbera/PODNet/blob/master/modules.py#L71

Add position embedding to input states.

Need to post-process the observation vector to extract (x,y) position and agent orientation for the minigrid-gym environment.

Add ability to leverage multiple GPUs for training by having a distributed trainig process (Similar to Hogwild, A3C etc.). Reference. Leveraging DistributedDataParallel for multiple gpus in a single node and multiple processes (each training process on a single node).

Revisit how true_next_state was calculated and inputted in the loss function. Instead of only states, should be something similar to state[i+1] (future state, and not current).

L_BC, L_ODC, Reg = loss_function(
    next_state_pred=next_state_pred,
    true_next_state=state,
    action_pred=action_pred,
    true_action=action,
    c_t=c_t)

Need a dataloader function for PODNet V2 to load trajectory tensors (in minibatch mode) and pad and split them into segment batches to be fed into the model.