nely-epfl / flygym Goto Github PK

... return odor intensity reading at antennae

It would be great to add more modular functions such as add_sensors, add_actuators, add_cameras, ... to make it easier for the user to find the right function to overwrite when writing a new class inheriting from NMF dedicated to a specific behavior.

Refactor the code keeping in mind that the environment could accommodate:

• More flies
• Other features than an arene (e.g ball, ...)

... so the unit is in mm and not um. The stiffness, damping, gravity, etc might also need to be tuned accordingly.

Looks like the fly model is not compatible with MuJoCo 3.0.0 and above because of a breaking change:

11. Removed mjOption.collision and the associated option/collision attribute.

See the changelog here.

... similar to what Victor did in PyBullet.

Upon creation of the NeuroMechFly instance, if the actuated_joints parameter does not contain all the controllable limb joints (when controlling a subset of joints is desired) a ValueError is raised in the line below. This variable seems to only be used for leg adhesion, so wrapping it under a conditional on whether leg adhesion is enabled fixes it, but I believe it should handle the case of unactuated leg joints? Or is there a better way to control a subset of joints?

... instead of passing a string like "flat" to NeuroMechFlyMuJoCo.__init__.

MuJoCo released its 3.0.0 version last week. There are a few API-breaking changes that made it incompatible with the MJCF file provided by FlyGym.

If you encounter an error that looks like the following upon loading the fly model, this is the reason:

self = MJCF Element: <option timestep="0.0001" gravity="0 0 -9810" integrator="Euler" solver="Newton" iterations="1000" tolerance="9.9999999999999998e-13" noslip_iterations="100" noslip_tolerance="1e-08" mpr_iterations="100"/>
attribute_name = 'collision'

    def _check_valid_attribute(self, attribute_name):
      if attribute_name not in self._spec.attributes:
>       raise AttributeError(
            '{!r} is not a valid attribute for <{}>'.format(
                attribute_name, self._spec.name))
E       AttributeError: Line 4: error while parsing element <option>: 'collision' is not a valid attribute for <option>

../miniconda3/envs/nmf/lib/python3.11/site-packages/dm_control/mjcf/element.py:534: AttributeError

Generally, I prefer to keep FlyGym compatible with the most up-to-date versions of its core dependencies such as MuJoCo. However, since this is a major update (2.x.x -> 3.x.x), I would like to spend more time verifying the compatibility and defer this effort for now.

A new release is made to address this issue.

I think it's a good idea to either

• remove the composite texture (is this how it's called?) in some body parts, especially legs, or
• find out how to set MuJoCo's random seed for generating these textures

They make the rendering results and vision-related things just slightly different. In visual navigation tasks this makes the whole behavior stochastic (because the slightly different visual input leads to slightly different descending drive and the difference just accumulates).

What do you think? @stimpfli

... with a sample 3D terrain type?

0.2.6

• TODO

0.2.5 (done)

• Docs for sensory arena — default size should be (300, 300)

Define a new class Pose. NeuroMechFlyMuJoCo.__init__ would receive a Pose object instead of a string like "default".

Gym actually specifies the format of .step()'s return values more strictly than I thought: https://gymnasium.farama.org/api/env/#gymnasium.Env.step

Namely it requires the following:

• observation (ObsType) – An element of the environment’s observation_space as the next observation due to the agent actions. An example is a numpy array containing the positions and velocities of the pole in CartPole.
• reward (SupportsFloat) – The reward as a result of taking the action.
• terminated (bool) – Whether the agent reaches the terminal state (as defined under the MDP of the task) which can be positive or negative. An example is reaching the goal state or moving into the lava from the Sutton and Barton, Gridworld. If true, the user needs to call reset().
• truncated (bool) – Whether the truncation condition outside the scope of the MDP is satisfied. Typically, this is a timelimit, but could also be used to indicate an agent physically going out of bounds. Can be used to end the episode prematurely before a terminal state is reached. If true, the user needs to call reset().
• info (dict) – Contains auxiliary diagnostic information (helpful for debugging, learning, and logging). This might, for instance, contain: metrics that describe the agent’s performance state, variables that are hidden from observations, or individual reward terms that are combined to produce the total reward. In OpenAI Gym <v26, it contains “TimeLimit.truncated” to distinguish truncation and termination, however this is deprecated in favour of returning terminated and truncated variables.

There are two solutions:

1. Keep doing it our way, and if we need to interface it with some existing framework/library (eg. stable-baselines3 for RL), we write and adapter.
2. Do it as specified above by Gym. This requires an API change but I think it's worth it. Concretely, this means:

• Currently, for the base environment, we are doing return obs, {} where the empty dict can be extended to contain arbitrary info.
• We need to change it to return obs, 0, False, False, {}. As before, the user can extend this class to implement different reward/termination criteria if desired.

If we opt for option 2 (which I personally prefer), we should do it ASAP (but perhaps after COBAR) before it becomes even more annoying. What do you think @stimpfli ?

For Isaac Gym I'm definitely going to stick with Gym's specified API above.

(this is a pro forma issue documenting a bug that's been fixed)

If the sliding friction (first number) is too high, then objects can run through each other even if collision is enabled.

• CPG, rule-based controller, hybrid controller, and turning can go into a single "locomotion" module
• Simple vision and connectome-based vision model can go into a single "vision" module
• Olfaction be its own thing (put it into a submodule for consistency)

Add a README for each submodule that describes what the files do.

1. Is the default setting of Black ok for everyone? I slightly prefer 80-column limit instead of 88, but maybe I should just buy a wider monitor.
2. Let's go through the demos before the next offering and make sure we stick with a good coding style. Maybe put some things into util modules.

In the long term, should we move away from dm_control and use just the mujoco Python binding, at least for the core simulation?

This would allow us to use GPU/TPU for the physics simulation using MJX that's been available since MuJoCo 3.0.0 (there's no plan on the dm_control side to retrofit MJX into dm_control). I doubt a morphologically complex model like NeuroMechFly can be run efficiently on the GPU (esp. given my experience with Isaac Gym), but I'd be curious to find out.

On the other hand, we can keep using dm_control's nice camera class for projecting from xyz coordinates to row-column coordinates on rendered image, or dm_control.mjcf for modifying the XML files, or add an interactive viewer with dm_control.viewer.

I can do this unless anyone else wants to help

It seems generally useful to be able to access the observation/state without having to supply an action and step the physics simulation. Let's make_get_observation a public method.

... so that we can feel free to call it as much as we want without running sensory preprocessing redundantly

eg. how direction is defined

Now that we're migrating away from PyBullet, I will only implement very limited functionality pro forma (just so it's easier to pick it up in case i need it in the future).

NeuroMechFLy.vision_update_mask is supposed to be a 1D binary array of length num_sim_steps, indicating whether the visual input is updated at each physics step.

However, currently the list that this mask array is generated from is appended to every time get_observation is called. Therefore the length of the array is the number of times the observation was queried, not the number of times the physics engine was advanced.

I've been using jupyter nbconvert to convert the demo notebooks to RST files for the website. This works OK but some manual finetuning of the RST files are required:

1. Video outputs are not embedded into the notebooks (otherwise they easily get way to large). I'm uploading the output media (videos, large images) to a separate repo so as not to clog this package, and adding them to the RST pages by linking to the "raw" URLs of these files on the separate repo. This needs to be done manually.
2. Sometimes image sizes need to be adjusted so that they display nicely on the website.
3. I added language specifiers to the code blocks to ensure proper Python syntax highlighting.
4. Something else? It's very human-in-the-loop...

This is quite annoying. Maybe we can automate this somehow? How about jupyter book? This is low priority but let's leave this issue here.

• Explore the usage of multi dog joints
• Clarify the relationship between dof sequence and inverse kinematics:
  • Either make it flexible to using any sequence
  • Or write script to change the order
• Remove actuators and cameras and initialise them in init()
• Check that simulation parameters are fully relevant
• Check that body, joint, inertial and geom parameters make sense

• Maze
• Vertical/overhanging walls
• Moving black ball

Helpful tool: export Fusion objects as URDF files https://github.com/syuntoku14/fusion2urdf

... that's easy to find in the index panel. The API ref for Fly and the kinematic replay tutorial can link to this page.

Hi, now that we have merged #134 (various recent changes into new-api) and renamed new-api to dev-v1.0.0, I want to make a laundry list of major tasks code-wise that we should take care of before releasing v1.0.0 and resubmitting the paper. Feel free to check these boxes as they are completed.

@sibocw:

• Test head stabilization in closed loop over complex terrain
• Improve fly following algorithm using not just T4/T5 for DN control; then merge head-stabilization-complex-terrain into dev-v1.0.0: #139
  • Follow-up: #150
    • Further follow-up: #153
• Merge dev-v1.0.0 into path-integration, test path integration over complex terrain using updated hybrid controller, and merge path-integration back into dev-v1.0.0 #141
  • Follow-up: #145
• Merge dev-cdf into dev-v1.0.0 #110 #151

@tkclam:

• Update RL stuff
• #140
• Merge or remove cobar-week4? not until COBAR is over

@stimpfli:

• Merge new-api-grooming into dev-v1.0.0
• Is new_steps already fully merged? If so, remove it
• Merge or remove fix_collisions_arena
• Is automated_adhesion already fully merged? If so, remove it

Contact force reading is currently passed as a pointer and not a copy, so the readings at past steps change unless the user made a copy explicitly. We should probably just return a copy.

Furthermore, it could be useful to return the contact forces as a (6, 3) array (6 legs, xyz) instead of an (18,) array.

This is a post hoc documentation of a little investigation.

I noticed that before and after #143, the contact force time series looks quite different (in magnitude at least).

Before (blue/orange = left/right legs):

After (note different y range):

@stimpfli failed to obtain the same results in a different notebook that he had been preparing for COBAR teaching. I looked into what might be going on.

Turns out our simulation configurations are a little different:

1. Calculation of total force: You summed forces experienced in each leg component in x, y, z separately before taking the norm. I took the norm experienced by each leg component first before summing them.
2. Simulation not long enough: The time series looks more like mine 10s into the simulation.
3. Spawn height of the fly: I changed it to the one I used but the impact of this is probably extremely minor in the end.
4. Correction weights: As discussed some weeks ago, I used the following for the hybrid turning controller, so that it’s essentially a CPG-only turning controller: correction_rates={"retraction": (0, 0), "stumbling": (0, 0)}

After unifying all of these, the new controller gives result like this (blue: @stimpfli's, orange, mine).

And the old ones (70b8b76) are like this:

Still not exactly the same, as I plotted my original time series (first figure) in the path integration simulation which is different from running forward walking for 10s. But I'm pretty confident that this is not an indication of an underlying bug now.

... as is the case in Isaac Gym.

1. End effector position and contact force sensor readings are not added to NeuroMechFlyMuJoCo.observation_space
2. They are not explained on the docs site

• Get rid of save_video_with_vision_insets and add the same logic to NeuroMechFly.save_video by default
• Do the same for olfaction
• Update the tutorials accordingly

... as it is being deprecated. Use other ways to find data files.

... or is this already the case?

• Interacting with NeuroMechFly
• Controlling locomotion with CPGs
• Rule-based controller
• Overcome complex terrain with a hybrid controller
• Encapsulating custom code into Gym environments: A turning controller
• Vision
• Olfaction

Please double check but only after #154 is done.

Problem the leading fly disappears under the floor after a while?? This is in the context of #147.

Video: https://github.com/NeLy-EPFL/flygym/assets/23410765/2320eac9-1bd1-4d9b-ad08-c3df03ecfa79
(The following fly starts from the wrong side of the arena in this example, but you can see the leading fly disappearing anyway.)

To reproduce:

git checkout fly-following-circle
git pull

# Establish baseline visual system activities
# Note: i'm running this in parallel, but doing so will lead to GPU out-of-memory error.
# Therefore I disabled the GPU by setting CUDA_VISIBLE_DEVICES to "".
CUDA_VISIBLE_DEVICES="" python flygym/examples/vision_connectome_model/response_to_fly.py

# Run closed-loop tracking script
python python flygym/examples/vision_connectome_model/follow_fly_closed_loop.py

Notes on follow_fly_closed_loop.py:

• It might be helpful to shorten the simulation, but you don't really see the problem unless it's 3s or longer
• Replace the last line (process_trial("blocks", True, (-5, -10))) with the line before that's commented out (Parallel(n_jobs=8)(delayed(process_trial)(*config) for config in configs)) to run multiple simulations in parallel. On my machine (126GB RAM), 8 processes was as high as I could reach before OOM. Set CUDA_VISIBLE_DEVICES too.
• If you don't want to run response_to_fly.py which establishes baseline activities of visual system neurons, you can download them here, decompress, and copy them under outputs/connectome_constrained_vision/baseline_response/:
  response_stats.zip

Feel free to work on fly-following-circle directly.

• Merge branch into main
• Update doc/source/changelog.rst
• Update version number in setup.py
• Make a release on GitHub
• Push to PyPI