This is the code accompanying the paper: "AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement Learning", published in IEEE INFOCOM 2022.

Mobile Crowdsensing (MCS) with smart devices has become an appealing paradigm for urban sensing.With the development of 5G-and-beyond technologies, unmanned aerial vehicles (UAVs) become possible for real-time applications, including wireless coverage, search and even disaster response. In this paper, we consider to use a group of UAVs as aerial base stations (BSs) to move around and collect data from multiple MCS users, forming a UAV crowdsensing campaign (UCS). Our goal is to maximize the collected data, geographical coverage whiling minimizing the age-of-information (AoI) of all mobile users simultaneously, with efficient use of constrained energy reserve. We propose a model-based deep reinforcement learning (DRL) framework called ”GCRL-min(AoI)”, which mainly consists of a novel model-based Monte Carlo tree search (MCTS) structure based on state-of-the- art approach MCTS (AlphaZero). We further improve it by adding a spatial UAV-user correlation extraction mechanism by a relational graph convolutional network (RGCN), and a next state prediction module to reduce the dependance of experience data. Extensive results and trajectory visualization on three real human mobility datasets in Purdue University, KAIST and NCSU show that GCRL-min(AoI) consistently outperforms five baselines, when varying different number of UAVs and maximum coupling loss in terms of four metrics.

1. Clone repo

   git clone https://github.com/BIT-MCS/GCRL-min-AoI.git
   cd GCRL-min-AoI

2. Install dependent packages

   # system-env
   sudo apt-get install libgeos++-dev libproj-dev
   
   # python-env
   conda create -n mcs python==3.8
   conda activate mcs
   conda install pytorch cudatoolkit tensorboard future
   conda install --channel conda-forge cartopy
   pip install -r requirements.txt
   
   # Install movingpandas
   mkdir requirements && cd requirements
   git clone https://github.com/anitagraser/movingpandas.git
   python setup.py develop

Train our solution

python train_our_policy.py --overwrite --output_dir logs/debug

Train our solution with trajectory visualization for debugs

python train_our_policy.py --overwrite --test_after_every_eval --vis_html --plot_loop --moving_line --output_dir logs/debug

Test with the trained models

python test_our_policy.py --vis_html --plot_loop --moving_line --model_dir logs/debug

Random test the env

python test_random.py --overwrite --vis_html --plot_loop --moving_line --output_dir logs/debug

• https://github.com/vita-epfl/CrowdNav
• https://github.com/ChanganVR/RelationalGraphLearning

This work is supported by the National Natural Science Foundation of China (No. 62022017).
Corresponding author: Chi Harold Liu.

If you have any question, please email [email protected].

If you are interested in our work, please cite our paper as

@inproceedings{dai2022aoi,
  author    = {Dai, Zipeng and Liu, Chi Harold and Ye, Yuxiao and Han, Rui and Yuan, Ye and Wang, Guoren and Tang, Jian},
  title     = {AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement Learning},
  booktitle = {IEEE International Conference on Computer Communications (INFOCOM)},
  year      = {2022},
}