ubcmocca / symmetricrl Goto Github PK

• new environments
  • non-locomotion
  • more symmetries: Ant, Sudoku
  • quadruped (raisim)
• symmetric gait
  • as a reward: r_t = | s_t - M(s_{t-T/2}) |
  • as a differentiable loss?
• revisiting PHASE for non-imitation tasks
  • better designed env
  • adaptible/foot strike based?
    • based on last foot strike side?
• unique net for each config

The Phase_Walker2DBulletEnv-v0 has an added artificial phase variable which might be useful to the other algorithms as well, so to be fair we might need to compare the others with phase as well.

• push final changes
• refactor/clean up if needed
• documentation
• make mocca_envs public
  • documentation for mocca_envs
  • clean up for mocca_envs

Current results aren't consistent with our expectations (and previous results):

Walker 2D

Walker 3D (fixed variance)

Walker 3D (learned variance)

• Though the net should have use a symmetric variance, so it's expected that it might not work.

Humanoid

Cassie 2D

Hi,

I'm a little confused about how to get mirrored obs/acts.
If I understand correctly, mirror_inds["sideneg_obs_inds"] should be the the index of obs that belongs to left or right side of a robot that need to be negated and swap when mirroring.
My problem is that only the obs of left side is included in mirror_inds["sideneg_obs_inds"] in the code, should be the corresponding right side also be included?
For example:
the mirroring function should fulfill obs = M(M(obs)).
if we have two corresponding left and right obs, say ob[left] = 1 and ob[right] = 2 and only "left" is in mirror_inds["sideneg_obs_inds"], after first mirroring ob[left] = 2 and ob[right] = -1, and mirroring them again, ob[left] = -1 and ob[right] = -2. It seems not correct. same thing for the action mirroring.

Looking forward to your responding!

• learning curves
  • 4 curves
  • same color across
  • ticks: time steps, Average Return
• tables for symmetry indices
• phase plots

We need to add an artificial phase variable to the base environments (no imitation) in order to use the phase-based method. Since the gait cycle length is fixed it can be constraining for this method so we need to tune the cycle length using either:

• hyper-parameter tuning
• looking at a learned motion and calculate the optimal cycle length based on that

Run all the methods with 5 random seed replicates each.
Environments:

• Walker2D
• Cassie2D
• Walker3D?
• Humanoid?
• Stepper?

Tune the w parameter in the auxiliary loss method.
Environments:

• Walker2D: 2
• Walker3D: 4
• Humanoid?
• Cassie2D: 4

• draw the phase plot for different environments
• calculate metric based on the phase plot similarity of left and right
  • foot strike based? different cycle lengths, how to find the middle of the cycle (half or optimize for min distance)

I need to prepare the presentation slides and content. What are the steps and the questions that need to be answered to do so?

• answer important questions
• prepare slides
• prepare video
• prepare the speech

• choose the set of videos
• make the video

Paper link

• methods
• page limit: 7-10
• template
• keywords and CCS
• experimental results
• metrics
• description of mirror functions
• net2?
• results section
• conference name, keywords, ...

Fetching submodule .environments
Already up to date.
Cloning into '/home/dyj/SymmetricRL/cassie_sim_to_real'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:zxieaa/cassie_sim_to_real.git' into submodule path '/home/dyj/SymmetricRL/cassie_sim_to_real' failed
Failed to clone 'cassie_sim_to_real'. Retry scheduled
Cloning into '/home/dyj/SymmetricRL/cassie_sim_to_real'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:zxieaa/cassie_sim_to_real.git' into submodule path '/home/dyj/SymmetricRL/cassie_sim_to_real' failed
Failed to clone 'cassie_sim_to_real' a second time, aborting

Deadline: Sep 20th

• re-write the results/summary section -> should be more clear and to the point
• fix typos
• publish code
• add the missing reference
• (optional) put phase-plot diagrams in the main paper
• (optional) tables to plots
• video
• clean-up code

Many other tasks have symmetry including pendulum, cartpole, etc. We experimented with some of them but never got to do a thorough comparison.