Code Monkey home page Code Monkey logo

crowdnav_dsrnn's Introduction

DSRNN_CrowdNav

This repository contains the codes for our paper titled "Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning" in ICRA 2021. For more details, please refer to the project website and arXiv preprint. For experiment demonstrations, please refer to the youtube video.

Please check out our more recent works in the following links:

Abstract

Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structural-Recurrent Neural Network (DS-RNN), a novel network that reasons about spatial and temporal relationships for robot decision making in crowd navigation. We train our network with model-free deep reinforcement learning without any expert supervision. We demonstrate that our model outperforms previous methods in challenging crowd navigation scenarios. We successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.

Setup

  1. Install Python3.6 (The code may work with other versions of Python, but 3.6 is highly recommended).
  2. Install the required python package using pip or conda. For pip, use the following command:
pip install -r requirements.txt

For conda, please install each package in requirements.txt into your conda environment manually and follow the instructions on the anaconda website.

  1. Install OpenAI Baselines.
git clone https://github.com/openai/baselines.git
cd baselines
pip install -e .
  1. Install Python-RVO2 library.

Getting started

This repository is organized in three parts:

  • crowd_sim/ folder contains the simulation environment. Details of the simulation framework can be found here.
  • crowd_nav/ folder contains configurations and non-neural network policies
  • pytorchBaselines/ contains the code for the DSRNN network and ppo algorithm.

Below are the instructions for training and testing policies.

Change configurations

  1. Environment configurations and training hyperparameters: modify crowd_nav/configs/config.py
  • For FoV environment (left in the figure below): change the value of robot.FOV in config.py
  • For Group environment (right in the figure below): set sim.group_human to True in config.py

Run the code

  1. Train a policy.
python train.py 
  1. Test policies.
    Please modify the test arguments in the begining of test.py.
    We provide two trained example weights for each type of robot kinematics:
    • Holonomic: data/example_model/checkpoints/27776.pt
    • Unicycle: data/example_model_unicycle/checkpoints/55554.pt
python test.py 
  1. Plot training curve.
python plot.py

(We only tested our code in Ubuntu 16.04 and 18.04 with Python 3.6.)

Learning curves

Learning curves of DS-RNN in 360 degrees FoV environment with 5 humans.

Citation

If you find the code or the paper useful for your research, please cite our paper:

@inproceedings{liu2020decentralized,
  title={Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning},
  author={Liu, Shuijing and Chang, Peixin and Liang, Weihang and Chakraborty, Neeloy and Driggs-Campbell, Katherine},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2021},
  pages={3517-3524}
}

Credits

Other contributors:
Peixin Chang
Neeloy Chakraborty

Part of the code is based on the following repositories:

[1] C. Chen, Y. Liu, S. Kreiss, and A. Alahi, “Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning,” in International Conference on Robotics and Automation (ICRA), 2019, pp. 6015–6022. (Github: https://github.com/vita-epfl/CrowdNav)

[2] I. Kostrikov, “Pytorch implementations of reinforcement learning algorithms,” https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail, 2018.

[3] A. Vemula, K. Muelling, and J. Oh, “Social attention: Modeling attention in human crowds,” in IEEE international Conference on Robotics and Automation (ICRA), 2018, pp. 1–7. (Github: https://github.com/jeanoh/big)

Contact

If you have any questions or find any bugs, please feel free to open an issue or pull request.

crowdnav_dsrnn's People

Contributors

evantancy avatar peixinc avatar shuijing725 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

crowdnav_dsrnn's Issues

Training problem

This project run twelve instances of the environment in parallel for collecting the robot's experenvces, but I want to use an environment for training, how can I do that?

Resume training from an existing checkpoint

Hi,I am very sorry to disturb you again.
I want to train from an existing checkpoint so I change the code in crowd_nav/configs/config.py as follow:
training.resume = True # resume training from an existing checkpoint or not
training.load_path = 'data/dummy/checkpoints/27776.pt' # if resume = True, load from the following checkpoint
But when I run train.py it shows error as follow:

File "/home/owen/文档/projects/CrowdNav/CrowdNav_DSRNN-main/CrowdNav_DSRNN-main/train.py", line 109, in main
actor_critic, _ = torch.load(load_path)
ValueError: too many values to unpack (expected 2)

I change the code in train.py as:
torch.load(load_path)
It run successfully.

Is it possible to use multiple robot?

Hi!

I'm still super new to Reinforcement Learning.

I want to know if it is possible to train multiple robots instead of just one like your code applied?

Thankyou!

Fov 2D visualization recreation help

hi,

I liked your work very much, i want to recreate your 2D animation of the FOV environment that you created. Please can you tell me how you did it?
where did you use RVO2 library in your code?
you said in that "For FoV environment : change the value of robot.FOV in config.py" , but change it to what?

thanks in advance :)

Algorithm Problem

Hi,

I read your code. I have some questions about the code details.
When using the ST-RNN, the masks are input into GRU as a parameter. Could you explain why you did this? What is the function of masks input?

Thanks!

Training parameters used

Hi there, I was just wondering what training parameters were used? Reading the comments from arguments.py, some default values in the argparse descriptions don't match the actual default values.

Thanks

Dense crowds

Hi guys how do I replicate the environment shown in your video with the dense crowds in circular formation?

Question about obeservation

HI @Shuijing725 !.
I have a single question about observation.
I saw your code in crowd_sim_dict.py.
In this script, generate_ob func was exist.
` def generate_ob(self, reset):
ob = {}

    # nodes
    visible_humans, num_visibles, human_visibility = self.get_num_human_in_fov()

    ob['robot_node'] = self.robot.get_full_state_list_noV()

    self.update_last_human_states(human_visibility, reset=reset)

    # edges
    # temporal edge: robot's velocity
    ob['temporal_edges'] = np.array([self.robot.vx, self.robot.vy])
    # spatial edges: the vector pointing from the robot position to each human's position
    ob['spatial_edges'] = np.zeros((self.human_num, 2))
    for i in range(self.human_num):
        relative_pos = np.array([self.last_human_states[i, 0] - self.robot.px, self.last_human_states[i, 1] - self.robot.py])
        ob['spatial_edges'][i] = relative_pos

    return ob`

So, i made observation above code.
` ob={}
ob['robot_node'] = np.array([self.cur_pos_x,self.cur_pos_y,
self.robot_radius,
self.goal_x,self.goal_y,
self.robot_max_vel,
self.robot_yaw])

        ob['temporal_edges'] = np.array([self.vel_x,self.vel_y]) ## robot vel x,y
        ob['spatial_edges'] = np.zeros((self.human_num, 2))
        for i in range(self.human_num):
            relative_pos = [self.human_state.mean_points[i].x, self.human_state.mean_points[i].y] ## already relative
            ob['spatial_edges'][i] = relative_pos
        `

And i referred that test.py and evaluate.py
` self.observation_space= ob
high = 1.2 * np.ones([2, ])
self.action_space = gym.spaces.Box(-high, high, dtype=np.float32)
net = self.Model(self.observation_space, self.action_space)
net.load_state_dict(torch.load(self.weight_path,map_location=torch.device('cpu')))

        eval_recurrent_hidden_states = {}
        node_num = 1
        edge_num = net.base.human_num + 1
        eval_recurrent_hidden_states['human_node_rnn'] = torch.zeros(1, node_num, self.config.SRNN.human_node_rnn_size * 1,
                                                            device="cpu")
        eval_recurrent_hidden_states['human_human_edge_rnn'] = torch.zeros(1, edge_num,
                                                                   self.config.SRNN.human_human_edge_rnn_size*1,
                                                            device="cpu")
        eval_masks = torch.zeros(1, 1, device='cpu')
        _, action, _, eval_recurrent_hidden_states = net.act(
                ob,
                eval_recurrent_hidden_states,
                eval_masks,
                deterministic=True)

`
after launched my code error occurred, like under script.

logger.warn( Exception in thread Thread-10: Traceback (most recent call last): File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/opt/ros/noetic/lib/python3/dist-packages/rospy/timer.py", line 237, in run self._callback(TimerEvent(last_expected, last_real, current_expected, current_real, last_duration)) File "drl_agent_node.py", line 147, in cbComputeActionArena _, action, _, eval_recurrent_hidden_states = net.act( File "/home/cai/sim_ws/src/navigation_sim/drl_nav_tool/drl_nav/train/Architectures/ppo_model.py", line 64, in act value, actor_features, rnn_hxs = self.base(inputs, rnn_hxs, masks, infer=True) File "/home/cai/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "/home/cai/sim_ws/src/navigation_sim/drl_nav_tool/drl_nav/train/Architectures/a2c_ppo_acktr/srnn_model.py", line 389, in forward robot_node = reshapeT(inputs['robot_node'], seq_length, nenv) File "/home/cai/sim_ws/src/navigation_sim/drl_nav_tool/drl_nav/train/Architectures/a2c_ppo_acktr/srnn_model.py", line 457, in reshapeT shape = T.size()[1:] AttributeError: 'list' object has no attribute 'size

I don't know how to solve this issue,,
What is problem in this part? . i think nothing was wrong to me...
Could you give me a small hint ?

There is no method named “set_policy()” in angent.py or human.py

Hi,thank you for your sharing! But I find some problem in the project.
In crowd_sim.py, the method named "randomize_human_policies()" needs a method of human named "set_policy()" to randomize the moving humans' policies to be either orca or social force,but I can’t find any method named “set_policy()” in angent.py or human.py. Maybe you didn't use it in your test so that you forgot it.

Implement multiple robot agents

hi,
I like your work very much,
I currently have an idea to do collaborative tasks among multiple robot agents.So I would like to ask you, how to implement multiple robot agents in the environment based on this github repository? Do you have any suggestions for this?

AssertionError: action space does not inherit from `gym.spaces.Space`

Hi!

I had following output when running the train.py file.

2023-03-05 17:18:49, INFO: Create other envs with new settings
Logging to C:\Users\Lab404\AppData\Local\Temp\openai-2023-03-05-17-18-49-452472
Creating dummy env object to get spaces
Traceback (most recent call last):
File "d:\LGR\RL\CrowdNav_DSRNN-main\CrowdNav_DSRNN-main\train.py", line 248, in
main()
File "d:\LGR\RL\CrowdNav_DSRNN-main\CrowdNav_DSRNN-main\train.py", line 88, in main
envs = make_vec_envs(env_name, config.env.seed, config.training.num_processes,
File "d:\LGR\RL\CrowdNav_DSRNN-main\CrowdNav_DSRNN-main\pytorchBaselines\a2c_ppo_acktr\envs.py", line 112, in make_vec_envs
envs = ShmemVecEnv(envs, context='spawn')
File "d:\LGR\RL\CrowdNav_DSRNN-main\CrowdNav_DSRNN-main\pytorchBaselines\a2c_ppo_acktr\shmem_vec_env.py", line 37, in init
dummy = env_fns[0]()
File "d:\LGR\RL\CrowdNav_DSRNN-main\CrowdNav_DSRNN-main\pytorchBaselines\a2c_ppo_acktr\envs.py", line 40, in _thunk
env = gym.make(env_id)
File "D:\downloads\PythonInstall\lib\site-packages\gym\envs\registration.py", line 669, in make
env = PassiveEnvChecker(env)
File "D:\downloads\PythonInstall\lib\site-packages\gym\wrappers\env_checker.py", line 23, in init
check_action_space(env.action_space)
File "D:\downloads\PythonInstall\lib\site-packages\gym\utils\passive_env_checker.py", line 74, in check_space
raise AssertionError(
AssertionError: action space does not inherit from 'gym.spaces.Space', actual type: <class 'NoneType'>

Note: I change envs = ShmemVecEnv(envs, context='fork') become envs = ShmemVecEnv(envs, context='spawn') because I had an error if I stick to "fork", and source on the internet said that fork is for MacOS, and I use Windows OS.

Do you have any idea on how to solve this?

Thank you!

The Accuracy of Model Training

Hello, I'm sorry to interrupt you. I used the parameters in /data/example_model/configs/config.py to train on my machine and got /data/dummy/checkpoints/27776.pt, and I used test.py to test my training results, the results are: TEST has success rate: 0.21, collision rate: 0.12, timeout rate: 0.67, NAV time: 20.78, total reward: 9.6464.
This is far from the 90% correct rate in your /data/example_model/test/test_27776.pt.log. Why is this?
My GPU is NVIDIA Tesla K80 and my CPU is three Xeon E5-2678 V3.
Python 3.7, CUDA 10.2, cuDNN 7.6, Pytorch 1.6.0, Ubuntu 18.04

2 agent scenarios

Hi I was wondering if it's possible to spawn a single human and a single robot for certain scenarios to evaluate side preference for passing, overtaking and crossing? I was trying to do that using the example model and it seems that the robot does well in approaching the goal but lingers around the last 0.5m of its goal. I've set sim.human_num = 1 and spawned the robot and human directly opposite on a circle for passing.

Unrecognized arguments

Hi, I tried to run test.py with parser(e.g. --test_case, --test_model), but it can't be done because error with unrecognized arguments.

Its shown usage like below:(seems like from argument.py)
usage: test.py [-h] [--env_config ENV_CONFIG] [--output_dir OUTPUT_DIR]
[--weights WEIGHTS] [--resume] [--load-path LOAD_PATH]
[--overwrite] [--num_threads NUM_THREADS] [--phase PHASE]
[--cuda-deterministic] [--no-cuda] [--seed SEED]
[--num-processes NUM_PROCESSES]
[--num-mini-batch NUM_MINI_BATCH] [--num-steps NUM_STEPS]
[--recurrent-policy] [--ppo-epoch PPO_EPOCH]

run on turtlebot2

hi,
I'm very interested in experiments on robots. What do I need to do to run on robots?
Thank you!

How to run in real world?

Hi, i read your paper and code. It was very interesting to me.

I have some question about running in real world.
In your paper, you used a camera for localization ( omit other camera).
But in your code ,there is just human position(px,py).
How to connect localization and velocity control(v,w)?

Could am i looking forward to your answer. Thank you!

fov environment

i want to creat fov environment like you with this code.

def render(self, mode='human', output_file=None):
from matplotlib import animation
import matplotlib.pyplot as plt
plt.rcParams['animation.ffmpeg_path'] = '/usr/bin/ffmpeg'

    x_offset = 0.11
    y_offset = 0.11
    cmap = plt.cm.get_cmap('hsv', 10)
    robot_color = 'yellow'
    goal_color = 'red'
    arrow_color = 'red'
    arrow_style = patches.ArrowStyle("->", head_length=4, head_width=2)

    if mode == 'human':
        fig, ax = plt.subplots(figsize=(7, 7))
        ax.set_xlim(-4, 4)
        ax.set_ylim(-4, 4)
        for human in self.humans:
            human_circle = plt.Circle(human.get_position(), human.radius, fill=False, color='b')
            ax.add_artist(human_circle)
        ax.add_artist(plt.Circle(self.robot.get_position(), self.robot.radius, fill=True, color='r'))
        plt.show()
    elif mode == 'traj':
        fig, ax = plt.subplots(figsize=(7, 7))
        ax.tick_params(labelsize=16)
        ax.set_xlim(-5, 5)
        ax.set_ylim(-5, 5)
        ax.set_xlabel('x(m)', fontsize=16)
        ax.set_ylabel('y(m)', fontsize=16)

        robot_positions = [self.states[i][0].position for i in range(len(self.states))]
        human_positions = [[self.states[i][1][j].position for j in range(len(self.humans))]
                           for i in range(len(self.states))]
        for k in range(len(self.states)):
            if k % 4 == 0 or k == len(self.states) - 1:
                robot = plt.Circle(robot_positions[k], self.robot.radius, fill=True, color=robot_color)
                humans = [plt.Circle(human_positions[k][i], self.humans[i].radius, fill=False, color=cmap(i))
                          for i in range(len(self.humans))]
                ax.add_artist(robot)
                for human in humans:
                    ax.add_artist(human)
            # add time annotation
            global_time = k * self.time_step
            if global_time % 4 == 0 or k == len(self.states) - 1:
                agents = humans + [robot]
                times = [plt.text(agents[i].center[0] - x_offset, agents[i].center[1] - y_offset,
                                  '{:.1f}'.format(global_time),
                                  color='black', fontsize=14) for i in range(self.human_num + 1)]
                for time in times:
                    ax.add_artist(time)
            if k != 0:
                nav_direction = plt.Line2D((self.states[k - 1][0].px, self.states[k][0].px),
                                           (self.states[k - 1][0].py, self.states[k][0].py),
                                           color=robot_color, ls='solid')
                human_directions = [plt.Line2D((self.states[k - 1][1][i].px, self.states[k][1][i].px),
                                               (self.states[k - 1][1][i].py, self.states[k][1][i].py),
                                               color=cmap(i), ls='solid')
                                    for i in range(self.human_num)]
                ax.add_artist(nav_direction)
                for human_direction in human_directions:
                    ax.add_artist(human_direction)
        plt.legend([robot], ['Robot'], fontsize=16)
        plt.show()
    elif mode == 'video':
        fig, ax = plt.subplots(figsize=(7, 7))
        ax.tick_params(labelsize=16)
        ax.set_xlim(-6, 6)
        ax.set_ylim(-6, 6)
        ax.set_xlabel('x(m)', fontsize=16)
        ax.set_ylabel('y(m)', fontsize=16)
        def calcFOVLineEndPoint(ang, point, extendFactor):
            # choose the extendFactor big enough
            # so that the endPoints of the FOVLine is out of xlim and ylim of the figure
            FOVLineRot = np.array([[np.cos(ang), -np.sin(ang), 0],
                                [np.sin(ang), np.cos(ang), 0],
                                [0, 0, 1]])
            point.extend([1])
            # apply rotation matrix
            newPoint = np.matmul(FOVLineRot, np.reshape(point, [3, 1]))
            # increase the distance between the line start point and the end point
            newPoint = [extendFactor * newPoint[0, 0], extendFactor * newPoint[1, 0], 1]
            return newPoint

        
        artists=[]

        # add goal
        goal=mlines.Line2D([self.robot.gx], [self.robot.gy], color=goal_color, marker='*', linestyle='None', markersize=15, label='Goal')
        ax.add_artist(goal)
        artists.append(goal)

        # add robot
        robotX,robotY=self.robot.get_position()

        robot=plt.Circle((robotX,robotY), self.robot.radius, fill=True, color=robot_color)
        ax.add_artist(robot)
        artists.append(robot)

        plt.legend([robot, goal], ['Robot', 'Goal'], fontsize=16)

        # compute orientation in each step and add arrow to show the direction
        radius = self.robot.radius
        arrowStartEnd=[]

        robot_theta = self.robot.theta if self.robot.kinematics == 'unicycle' else np.arctan2(self.robot.vy, self.robot.vx)

        arrowStartEnd.append(((robotX, robotY), (robotX + radius * np.cos(robot_theta), robotY + radius * np.sin(robot_theta))))

        for i, human in enumerate(self.humans):
            theta = np.arctan2(human.vy, human.vx)
            arrowStartEnd.append(((human.px, human.py), (human.px + radius * np.cos(theta), human.py + radius * np.sin(theta))))

        arrows = [patches.FancyArrowPatch(*arrow, color=arrow_color, arrowstyle=arrow_style)
                for arrow in arrowStartEnd]
        for arrow in arrows:
            ax.add_artist(arrow)
            artists.append(arrow)


        # draw FOV for the robot
        # add robot FOV
        if self.robot_fov < np.pi * 2:
            FOVAng = self.robot_fov / 2
            FOVLine1 = mlines.Line2D([0, 0], [0, 0], linestyle='--')
            FOVLine2 = mlines.Line2D([0, 0], [0, 0], linestyle='--')


            startPointX = robotX
            startPointY = robotY
            endPointX = robotX + radius * np.cos(robot_theta)
            endPointY = robotY + radius * np.sin(robot_theta)

            # transform the vector back to world frame origin, apply rotation matrix, and get end point of FOVLine
            # the start point of the FOVLine is the center of the robot
            FOVEndPoint1 = calcFOVLineEndPoint(FOVAng, [endPointX - startPointX, endPointY - startPointY], 20. / self.robot.radius)
            FOVLine1.set_xdata(np.array([startPointX, startPointX + FOVEndPoint1[0]]))
            FOVLine1.set_ydata(np.array([startPointY, startPointY + FOVEndPoint1[1]]))
            FOVEndPoint2 = calcFOVLineEndPoint(-FOVAng, [endPointX - startPointX, endPointY - startPointY], 20. / self.robot.radius)
            FOVLine2.set_xdata(np.array([startPointX, startPointX + FOVEndPoint2[0]]))
            FOVLine2.set_ydata(np.array([startPointY, startPointY + FOVEndPoint2[1]]))

            ax.add_artist(FOVLine1)
            ax.add_artist(FOVLine2)
            artists.append(FOVLine1)
            artists.append(FOVLine2)

        # add humans and change the color of them based on visibility
        human_circles = [plt.Circle(human.get_position(), human.radius, fill=False) for human in self.humans]


        for i in range(len(self.humans)):
            ax.add_artist(human_circles[i])
            artists.append(human_circles[i])

            # green: visible; red: invisible
            if self.detect_visible(self.robot, self.humans[i], robot1=True):
                human_circles[i].set_color(c='g')
            else:
                human_circles[i].set_color(c='r')
            plt.text(self.humans[i].px - 0.1, self.humans[i].py - 0.1, str(i), color='black', fontsize=12)


        plt.pause(0.1)
        for item in artists:
            item.remove() # there should be a better way to do this. For example,
            # initially use add_artist and draw_artist later on
        for t in ax.texts:
            t.set_visible(False)

        # add robot and its goal
        robot_positions = [state[0].position for state in self.states]
        # draw a star at the goal position (0,4)
        goal = mlines.Line2D([0], [4], color=goal_color, marker='*', linestyle='None', markersize=15, label='Goal')
        robot = plt.Circle(robot_positions[0], self.robot.radius, fill=True, color=robot_color)
        ax.add_artist(robot)
        ax.add_artist(goal)
        plt.legend([robot, goal], ['Robot', 'Goal'], fontsize=16, numpoints=1)   # numpoints=1: only 1 star in the legend

        # add humans and their numbers
        human_positions = [[state[1][j].position for j in range(len(self.humans))] for state in self.states]
        humans = [plt.Circle(human_positions[0][i], self.humans[i].radius, fill=False)
                  for i in range(len(self.humans))]
        human_numbers = [plt.text(humans[i].center[0] - x_offset, humans[i].center[1] - y_offset, str(i),
                                  color='black', fontsize=12) for i in range(len(self.humans))]
        for i, human in enumerate(humans):
            ax.add_artist(human)
            ax.add_artist(human_numbers[i])

        # add time annotation
        time = plt.text(-1, 5, 'Time: {}'.format(0), fontsize=16)
        ax.add_artist(time)

        # compute attention scores
        if self.attention_weights is not None:
            attention_scores = [
                plt.text(-6, 6 - 1.0 * i, 'Human {}: {:.2f}'.format(i + 1, self.attention_weights[0][i]),
                         fontsize=16) for i in range(len(self.humans))]

        # compute orientation in each step and use arrow to show the direction
        radius = self.robot.radius
        if self.robot.kinematics == 'unicycle':
            orientation = [((state[0].px, state[0].py), (state[0].px + radius * np.cos(state[0].theta),
                                                         state[0].py + radius * np.sin(state[0].theta))) for state
                           in self.states]
            orientations = [orientation]
        else:
            orientations = []
            for i in range(self.human_num + 1):
                orientation = []
                for state in self.states:
                    if i == 0:
                        agent_state = state[0]
                    else:
                        agent_state = state[1][i - 1]
                    theta = np.arctan2(agent_state.vy, agent_state.vx)
                    orientation.append(((agent_state.px, agent_state.py), (agent_state.px + radius * np.cos(theta),
                                         agent_state.py + radius * np.sin(theta))))
                orientations.append(orientation)
        arrows = {}
        arrows[0] = [patches.FancyArrowPatch(*orientation[0], color=arrow_color, arrowstyle=arrow_style)
                  for orientation in orientations]
        for arrow in arrows[0]:
            ax.add_artist(arrow)
        global_step = {}
        global_step[0] = 0

        def update(frame_num):
            # nonlocal global_step
            # nonlocal arrows
            global_step[0] = frame_num
            robot.center = robot_positions[frame_num]
            for i, human in enumerate(humans):
                human.center = human_positions[frame_num][i]
                human_numbers[i].set_position((human.center[0] - x_offset, human.center[1] - y_offset))
                for arrow in arrows[0]:
                    arrow.remove()
                arrows[0] = [patches.FancyArrowPatch(*orientation[frame_num], color=arrow_color,
                                                  arrowstyle=arrow_style) for orientation in orientations]
                for arrow in arrows[0]:
                    ax.add_artist(arrow)
                if self.attention_weights is not None:
                    human.set_color(str(self.attention_weights[frame_num][i]))
                    attention_scores[i].set_text('human {}: {:.2f}'.format(i, self.attention_weights[frame_num][i]))

            time.set_text('Time: {:.2f}'.format(frame_num * self.time_step))

        def plot_value_heatmap():
            assert self.robot.kinematics == 'holonomic'
            # when any key is pressed draw the action value plot
            fig, axis = plt.subplots()
            speeds = self.robot.policy.speeds
            rotations = self.robot.policy.rotations + [np.pi * 2]
            r, th = np.meshgrid(speeds, rotations)
            z = np.array(self.action_values[global_step[0] % len(self.states)][1:])
            z = (z - np.min(z)) / (np.max(z) - np.min(z))  # z: normalized action values
            z = np.append(z, z[:6])
            z = z.reshape(16, 6)  # rotations: 16   speeds:6
            polar = plt.subplot(projection="polar")
            polar.tick_params(labelsize=16)
            mesh = plt.pcolormesh(th, r, z, cmap=plt.cm.viridis)
            plt.plot(rotations, r, color='k', ls='none')
            plt.grid()
            cbaxes = fig.add_axes([0.85, 0.1, 0.03, 0.8])
            cbar = plt.colorbar(mesh, cax=cbaxes)
            cbar.ax.tick_params(labelsize=16)
            plt.show()

        def on_click(event):
            anim.running ^= True
            if anim.running:
                anim.event_source.stop()
                if hasattr(self.robot.policy, 'action_values'):
                    plot_value_heatmap()
            else:
                anim.event_source.start()

        fig.canvas.mpl_connect('key_press_event', on_click)
        anim = animation.FuncAnimation(fig, update, frames=len(self.states), interval=self.time_step * 1000)
        anim.running = True
        anim.save('testcase_animation.gif', writer='imagemagick')

        if output_file is not None:
            ffmpeg_writer = animation.writers['ffmpeg']
            writer = ffmpeg_writer(fps=8, metadata=dict(artist='Me'), bitrate=1800)
            anim.save(output_file, writer=writer)
        else:
            plt.show()
    else:
        raise NotImplementedError

i'm trying to make fov animation with your code and can you help to modify the code for fov? how can i make animation?

Algorithm Problem

Hi, sorry to bother you again! This experiment uses the PPO algorithm to update the policy, but this is an online policy. Can I use an offline policy, such as SAC, to replace the PPO algorithm in my experiment?

Dynamic changing number of humans

Hi, I wonder this simulation can be applied to the dynamic changing number of people.

Since the hidden states and spatial edges are dependent on human_num, it seems to cause a problem in the forward() of srnn_model.py if the number of people changes at every step.

Where am I missing something?

Thanks,

evaluation.py

Hi I was just wondering in evaluation.py, should it be obs['robot_node'][0, 0, 0:2] instead of obs['robot_node'][0, 0, 1:3] to get x,y position?
Based off line 49 in crowd_sim_dict.py: ob['robot_node'] = self.robot.get_full_state_list_noV()

Also is chc the total change in heading for an episode?

plot trajectory

Hi
In the code crowdnav in the evaluation section, there is a trajectory display of people's movements, but in your code, this section has been removed and you have also used it in your article. How can we show a trajectory?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.