Code Monkey home page Code Monkey logo

Comments (12)

beyretb avatar beyretb commented on June 12, 2024 1

Hi,

Sorry for the late response, I'm travelling these days and I wanted to test the solution below quite extensively. I have played around with both the source and the code available to you, on several machines including the evaluation p2 instances (AWS), and here is what I came up with:

  • The speed is machine independent, no matter what resources you use, the same actions in the same environment will give the same speed on any machine (which should be kind of obvious, but wasn't at all from the Unity documentation)
  • The speed is also independent from running the environment in training or inference mode (same actions give the same speeds)
  • As you mention this clearly is a tricky parameter to use as it is computed by Unity's physics engine and comes from the various forces applied to the agent. I couldn't find a clear formula for computing the speed manually (my guess is that the drag applied to the agent complicates the calculations).
  • But using the example code above I managed to find that the "time coefficient" (not sure how to call this, but basically what should be the time in distance=speed*time) is somewhere between 0.059 and 0.6

Therefore, in the example above, replace:

delta_distance = step_time_length * speed[0, 2]

with:

delta_distance = 0.0595 * speed[0, 2]

and you should end up with approximately 39.

I am aware this is not ideal, and I will keep trying to get you a closed formula for this, but in the meantime I can confirm that all of this is machine and running-mode independent, and that we have an approximation of the time factor. I hope all of this helps!

from animalai-olympics.

MLanghof avatar MLanghof commented on June 12, 2024 1

Hi @beyretb,

Thank you for your information and tests!

We have computed the same time step constant as you from the data @dan9thsense provided (noticing that the measured speeds stayed exactly the same in those runs, even with wall clock times occasionally spiking). It is very reassuring that these remain the same on different machines and for different environment settings.

In terms of the physics this is all the information I need, I personally don't think a closed form for the speed is necessary.

Thanks again!

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

experimentally, looks like about 0.1 second for each step.

from animalai-olympics.

beyretb avatar beyretb commented on June 12, 2024

Hello,

The time used is the same as your machine's clock, meaning "our time". I wrote a script for you to visualise this: let's create an arena with just one agent, make it move forward until it hits a wall and compute how far it travelled using distance = speed*time.

First, add the below YAML configuration to a new file examples/configs/empty.yaml:

!ArenaConfig
arenas:
  0: !Arena
    t: 0
    items:
    - !Item
      name: Agent
      positions:
        - !Vector3 {x: 20, y: 0.5, z: 0.5}
      rotations: [0]

Then we can create a script that runs in inference mode and simply tells the agent to go forward. Add the code below in a new file examples/distance.py:

from animalai.envs.environment import UnityEnvironment
from animalai.envs.arena_config import ArenaConfig
import time

env = UnityEnvironment(
    n_arenas=1,
    file_name='../env/AnimalAI',
    worker_id=1,
    seed=0,
    docker_training=False,
    inference=True
)

arena_config_in = ArenaConfig('configs/empty.yaml')
env.reset(arenas_configurations=arena_config_in)
total_distance = 0
try:
    while True:
        start = time.time()     # start a timer before taking a step
        res = env.step([1, 0])  # send a forward action to the environment
        step_time_length = time.time() - start      # compute the time it took to take the step
        speed = res['Learner'].vector_observations
        total_distance += step_time_length * speed[0, 2]    # compute the distance covered in one step
        if speed[0, 2] == 0:
            #   if our agent reached the other end of the arena we close the environment
            print('Distance covered by the agent {}'.format(total_distance))
            raise KeyboardInterrupt
except KeyboardInterrupt:
    env.close()

Running this script gives me a distance of 39.08, which is consistent with the agent starting in z=0.5 and ending in z=49.5 (the agent is a sphere of radius 0.5). I think the extra 0.08 come from extra overhead computation on top of just rendering the frames.

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

Weird, that script gives me variable distances between about 3 and 5 meters. To help figure it out, I then modified it to print out values for speed, delta time, delta distance and total distance at each step. Note that I renamed the empty.yaml file to distance_test.yaml

Here is the modified script:

from animalai.envs.environment import UnityEnvironment
from animalai.envs.arena_config import ArenaConfig
import time

env = UnityEnvironment(
        n_arenas=1,  # Change this to train on more arenas
        file_name='../env/AnimalAI',
        worker_id=1,
        seed=10,
        docker_training=False,
        play=False
    )

arena_config_in = ArenaConfig('configs/distance_test.yaml')
env.reset(arenas_configurations=arena_config_in)
total_distance = 0
try:
    while True:
        start = time.time()     # start a timer before taking a step
        res = env.step([1, 0])  # send a forward action to the environment
        step_time_length = time.time() - start      # compute the time it took to take the step
        speed = res['Learner'].vector_observations
        delta_distance = step_time_length * speed[0, 2]    # compute the distance covered in one step
        total_distance += delta_distance
        print("speed = {0:.4f}, delta_time = {1:.4f}, delta_distance = {2:.4f}, total_distance = {3:.4f}".format(speed[0,2],
             step_time_length, delta_distance, total_distance))
        if speed[0, 2] == 0:
            #   if our agent reached the other end of the arena we close the environment
            print('Distance covered by the agent {}'.format(total_distance))
            raise KeyboardInterrupt
except KeyboardInterrupt:
    env.close()

And here are the results:

INFO:mlagents.envs:
'Academy' started successfully!
Unity Academy name: Academy
        Number of Brains: 1
        Number of Training Brains : 1
speed = -0.0000, delta_time = 0.0140, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0042, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = 1.6380, delta_time = 0.0045, delta_distance = 0.0074, total_distance = 0.0074
speed = 3.8890, delta_time = 0.0040, delta_distance = 0.0156, total_distance = 0.0230
speed = 5.8814, delta_time = 0.0086, delta_distance = 0.0508, total_distance = 0.0738
speed = 7.6442, delta_time = 0.0041, delta_distance = 0.0312, total_distance = 0.1050
speed = 9.2038, delta_time = 0.0039, delta_distance = 0.0357, total_distance = 0.1407
speed = 10.5836, delta_time = 0.0043, delta_distance = 0.0458, total_distance = 0.1864
speed = 11.8043, delta_time = 0.0045, delta_distance = 0.0535, total_distance = 0.2399
speed = 12.8844, delta_time = 0.0040, delta_distance = 0.0511, total_distance = 0.2910
speed = 13.8400, delta_time = 0.0040, delta_distance = 0.0553, total_distance = 0.3463
speed = 14.6854, delta_time = 0.0042, delta_distance = 0.0611, total_distance = 0.4074
speed = 15.4334, delta_time = 0.0041, delta_distance = 0.0631, total_distance = 0.4705
speed = 16.0952, delta_time = 0.0039, delta_distance = 0.0635, total_distance = 0.5339
speed = 16.6807, delta_time = 0.0040, delta_distance = 0.0670, total_distance = 0.6010
speed = 17.1987, delta_time = 0.0040, delta_distance = 0.0688, total_distance = 0.6698
speed = 17.6570, delta_time = 0.0040, delta_distance = 0.0708, total_distance = 0.7406
speed = 18.0624, delta_time = 0.0043, delta_distance = 0.0768, total_distance = 0.8174
speed = 18.4212, delta_time = 0.0039, delta_distance = 0.0726, total_distance = 0.8900
speed = 18.7386, delta_time = 0.0042, delta_distance = 0.0793, total_distance = 0.9693
speed = 19.0194, delta_time = 0.0040, delta_distance = 0.0769, total_distance = 1.0462
speed = 19.2678, delta_time = 0.0042, delta_distance = 0.0800, total_distance = 1.1263
speed = 19.4876, delta_time = 0.0041, delta_distance = 0.0794, total_distance = 1.2056
speed = 19.6821, delta_time = 0.0686, delta_distance = 1.3510, total_distance = 2.5566
speed = 19.8541, delta_time = 0.0054, delta_distance = 0.1067, total_distance = 2.6633
speed = 20.0063, delta_time = 0.0047, delta_distance = 0.0944, total_distance = 2.7578
speed = 20.1410, delta_time = 0.0048, delta_distance = 0.0965, total_distance = 2.8543
speed = 20.2602, delta_time = 0.0048, delta_distance = 0.0980, total_distance = 2.9523
speed = 20.3656, delta_time = 0.0045, delta_distance = 0.0926, total_distance = 3.0449
speed = 20.4588, delta_time = 0.0042, delta_distance = 0.0867, total_distance = 3.1317
speed = 20.5414, delta_time = 0.0043, delta_distance = 0.0884, total_distance = 3.2200
speed = 20.6144, delta_time = 0.0052, delta_distance = 0.1081, total_distance = 3.3281
speed = 20.6789, delta_time = 0.0051, delta_distance = 0.1048, total_distance = 3.4330
speed = 20.7361, delta_time = 0.0048, delta_distance = 0.0993, total_distance = 3.5323
speed = 20.7867, delta_time = 0.0047, delta_distance = 0.0968, total_distance = 3.6291
speed = 20.8314, delta_time = 0.0062, delta_distance = 0.1290, total_distance = 3.7581
speed = 20.8710, delta_time = 0.0046, delta_distance = 0.0954, total_distance = 3.8535
speed = 20.9060, delta_time = 0.0046, delta_distance = 0.0958, total_distance = 3.9493
speed = 20.9370, delta_time = 0.0065, delta_distance = 0.1367, total_distance = 4.0859
speed = 20.9644, delta_time = 0.0046, delta_distance = 0.0968, total_distance = 4.1828
speed = 20.9886, delta_time = 0.0054, delta_distance = 0.1133, total_distance = 4.2960
speed = 0.0000, delta_time = 0.0050, delta_distance = 0.0000, total_distance = 4.2960
Distance covered by the agent 4.296039030300335

next run:

speed = -0.0000, delta_time = 0.0142, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0042, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0042, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = -0.0000, delta_time = 0.0040, delta_distance = -0.0000, total_distance = -0.0000
speed = 1.6380, delta_time = 0.0045, delta_distance = 0.0073, total_distance = 0.0073
speed = 3.8890, delta_time = 0.0040, delta_distance = 0.0156, total_distance = 0.0229
speed = 5.8814, delta_time = 0.0042, delta_distance = 0.0249, total_distance = 0.0479
speed = 7.6442, delta_time = 0.0047, delta_distance = 0.0356, total_distance = 0.0834
speed = 9.2038, delta_time = 0.0039, delta_distance = 0.0357, total_distance = 0.1191
speed = 10.5836, delta_time = 0.0038, delta_distance = 0.0407, total_distance = 0.1597
speed = 11.8043, delta_time = 0.0044, delta_distance = 0.0520, total_distance = 0.2118
speed = 12.8844, delta_time = 0.0040, delta_distance = 0.0514, total_distance = 0.2632
speed = 13.8400, delta_time = 0.0040, delta_distance = 0.0558, total_distance = 0.3190
speed = 14.6854, delta_time = 0.0042, delta_distance = 0.0613, total_distance = 0.3803
speed = 15.4334, delta_time = 0.0040, delta_distance = 0.0625, total_distance = 0.4428
speed = 16.0952, delta_time = 0.0039, delta_distance = 0.0629, total_distance = 0.5056
speed = 16.6807, delta_time = 0.0041, delta_distance = 0.0676, total_distance = 0.5732
speed = 17.1987, delta_time = 0.0044, delta_distance = 0.0758, total_distance = 0.6490
speed = 17.6570, delta_time = 0.0040, delta_distance = 0.0711, total_distance = 0.7201
speed = 18.0624, delta_time = 0.0041, delta_distance = 0.0733, total_distance = 0.7934
speed = 18.4212, delta_time = 0.0040, delta_distance = 0.0739, total_distance = 0.8672
speed = 18.7386, delta_time = 0.0041, delta_distance = 0.0774, total_distance = 0.9447
speed = 19.0194, delta_time = 0.0040, delta_distance = 0.0764, total_distance = 1.0210
speed = 19.2678, delta_time = 0.0042, delta_distance = 0.0807, total_distance = 1.1018
speed = 19.4876, delta_time = 0.0041, delta_distance = 0.0791, total_distance = 1.1809
speed = 19.6821, delta_time = 0.0170, delta_distance = 0.3350, total_distance = 1.5158
speed = 19.8541, delta_time = 0.0044, delta_distance = 0.0875, total_distance = 1.6033
speed = 20.0063, delta_time = 0.0047, delta_distance = 0.0943, total_distance = 1.6975
speed = 20.1410, delta_time = 0.0041, delta_distance = 0.0828, total_distance = 1.7803
speed = 20.2602, delta_time = 0.0043, delta_distance = 0.0863, total_distance = 1.8666
speed = 20.3656, delta_time = 0.0044, delta_distance = 0.0892, total_distance = 1.9558
speed = 20.4588, delta_time = 0.0043, delta_distance = 0.0876, total_distance = 2.0435
speed = 20.5414, delta_time = 0.0043, delta_distance = 0.0893, total_distance = 2.1328
speed = 20.6144, delta_time = 0.0044, delta_distance = 0.0910, total_distance = 2.2238
speed = 20.6789, delta_time = 0.0043, delta_distance = 0.0894, total_distance = 2.3132
speed = 20.7361, delta_time = 0.0045, delta_distance = 0.0943, total_distance = 2.4075
speed = 20.7867, delta_time = 0.0043, delta_distance = 0.0901, total_distance = 2.4976
speed = 20.8314, delta_time = 0.0045, delta_distance = 0.0943, total_distance = 2.5919
speed = 20.8710, delta_time = 0.0048, delta_distance = 0.0994, total_distance = 2.6913
speed = 20.9060, delta_time = 0.0048, delta_distance = 0.0999, total_distance = 2.7911
speed = 20.9370, delta_time = 0.0050, delta_distance = 0.1047, total_distance = 2.8958
speed = 20.9644, delta_time = 0.0044, delta_distance = 0.0920, total_distance = 2.9878
speed = 20.9886, delta_time = 0.0052, delta_distance = 0.1084, total_distance = 3.0962
speed = 0.0000, delta_time = 0.0046, delta_distance = 0.0000, total_distance = 3.0962
Distance covered by the agent 3.096249973786115

from animalai-olympics.

beyretb avatar beyretb commented on June 12, 2024

Could you kindly run the exact code I wrote for you and let me know the output please?

In the one you modified above you changed the parameters of the environment which change the rendering speed in Unity, therefore your outputs and mine are not comparable.

I think that what happens in your case (play=False) is that Unity optimises for frames per second and I'm not sure how this impact the time it takes to call env.step (it might not be very relevant to compute this running time then). I will ask Unity anyway, but please run the code I gave you so that we can at least make sure you get similar results.

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

Oh, I didn't realize setting play=False would matter. I thought that was needed when doing training, so it is part of my standard setup. Does that mean we cannot get reliable distances during training? To be able to use distance, it would have to calculate to the same number during both training and running the agent.

Using your script, I get a value of around 41.8. It varies by about +- 0.1. However, after running it a couple times, it generates an error and after that will no longer run.

Traceback (most recent call last):
  File "~/AnimalAI-Olympics/animalai/animalai/envs/rpc_communicator.py", line 70, in check_port
    s.bind(("localhost", port))
OSError: [Errno 98] Address already in use

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/AnimalAI-Olympics/examples/test_distance.py", line 11, in <module>
    inference=True
  File "~/AnimalAI-Olympics/animalai/animalai/envs/environment.py", line 59, in __init__
    self.communicator = self.get_communicator(worker_id, base_port)
  File "~/AnimalAI-Olympics/animalai/animalai/envs/environment.py", line 235, in get_communicator
    return RpcCommunicator(worker_id, base_port)
  File "~/AnimalAI-Olympics/animalai/animalai/envs/rpc_communicator.py", line 43, in __init__
    self.create_server()
  File "~/AnimalAI-Olympics/animalai/animalai/envs/rpc_communicator.py", line 49, in create_server
    self.check_port(self.port)
  File "~/AnimalAI-Olympics/animalai/animalai/envs/rpc_communicator.py", line 72, in check_port
    raise UnityWorkerInUseException(self.worker_id)
animalai.envs.exception.UnityWorkerInUseException: Couldn't start socket communication because worker number 1 is still in use. You may need to manually close a previously opened environment or use a different worker number.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "~/AnimalAI-Olympics/animalai/animalai/envs/environment.py", line 428, in _close
    self.communicator.close()
AttributeError: 'UnityEnvironment' object has no attribute 'communicator'

Process finished with exit code 1

from animalai-olympics.

beyretb avatar beyretb commented on June 12, 2024

Does that mean we cannot get reliable distances during training?

I would tend to say you cannot no, the speeds are correct, but the time it takes to run env.step() might be too noisy to capture the actual time it takes to render the frames (which is where the speed comes from). In all fairness, we included the speed as a proxy for motion senses animals may have, even though we as human feel acceleration rather than speed. It was not meant to compute distances, it's not really a skill animals have in the real world :) But you're of course free to use this the way you want!

The communicator issue is unrelated, it is due to the environment not closing properly and leaving the communication port open therefore preventing a new connection to happen. One solution is to wait for the port to close, another is to close it manually, but the easiest is probably to randomise the worker_id (as done here)

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

I think I do have sense of distance, even walking with my eyes closed, that is part of making an internal map of my environment. But OK, not available for the agent through the velocities. The link is broken.

from animalai-olympics.

beyretb avatar beyretb commented on June 12, 2024

Yes, I definitely agree we do have an internal representation for that. My statement was poorly worded: I just meant that we don't directly compute speed*time to get the distance we cover sadly.

To come back on computing the distance, the velocity does not depend on the machine your code runs on, but the time it takes to execute a step does (hence this time being a bad measure). Therefore you could instead compute the distance covered in a single step by playing around with the code above for example.

Github was experiencing issues between your post and mine, try the link again it should be fine.

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

Link is good now.

I guess I could get the timing figured out for a specific machine, but it won't work for the submitted entries since those will run on some different machine, right? Also, since it would be different in training vs running trained networks, there would have to be some sort of calibration value to go from to the other. In general it seems like a fragile parameter, one that would not be reliable enough to depend on for decisions.

from animalai-olympics.

dan9thsense avatar dan9thsense commented on June 12, 2024

Thanks for figuring this out.

from animalai-olympics.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.