jordanmicahbennett / packnet-sfm Goto Github PK

This repository contains code for the following papers:

Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos and Adrien Gaidon

[Full paper] [YouTube]

Vitor Guizilini, Jie Li, Rares Ambrus, Sudeep Pillai and Adrien Gaidon

[Full paper] [YouTube]

Rares Ambrus, Vitor Guizilini, Jie Li, Sudeep Pillai and Adrien Gaidon

[Full paper]

Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus and Adrien Gaidon

[Full paper]

Sudeep Pillai, Rares Ambrus and Adrien Gaidon

[Full paper] [YouTube]

• PackNet: A new convolutional network architecture for high-resolution self-supervised monocular depth estimation. We propose new packing and unpacking blocks that jointly leverage 3D convolutions to learn representations that maximally propagate dense appearance and geometric information while still being able to run in real time.

• Weak Velocity Supervision: A novel optional loss that can leverage the camera’s velocity when available (e.g. from cars, robots, mobile phones) to solve the inherent scale ambiguity in monocular vision.

• Dense Depth for Automated Driving (DDAD): A new dataset that leverages diverse logs from a fleet of well-calibrated self-driving cars equipped with cameras and high-accuracy long-range LiDARs. Compared toexisting benchmarks, DDAD enables much more accurate depth evaluation at range, which is key for high resolution monocular depth estimation methods.

>

git clone https://github.com/vguizilini/packnet-sfm.git

sudo ln -s path/to/data /data

• For convenience, we also provide the pre-computed depth maps used in our papers (unzip into the same root folder)

  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/depth_maps/KITTI_raw_velodyne.tar.gz
  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/depth_maps/KITTI_raw_groundtruth.tar.gz
  

• Self-Supervised (192x640, K)

  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/models/PackNet_MR_selfsup_K.pth.tar
  

• Self-Supervised (192x640, CS)

  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/models/PackNet_MR_selfsup_CS.pth.tar
  

• Self-Supervised Scale-Aware (192x640, CS → K)

  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/models/PackNet_MR_velsup_CStoK.pth.tar
  

• Semi-Supervised (Annotated depth maps) (192x640, CS → K)

  wget https://tri-ml-public.s3.amazonaws.com/github/packnet-sfm/models/PackNet_MR_semisup_CStoK.pth.tar
  

bash evaluate_kitti.sh

The source code is released under the MIT license.

Depending on the application, please use the following citations when referencing our work:

@misc{packnet-sfm-selfsup,
  author = {Vitor Guizilini and Rares Ambrus and Sudeep Pillai and Allan Raventos and Adrien Gaidon},
  title = {3D Packing for Self-Supervised Monocular Depth Estimation},
  archivePrefix = {arXiv:1905.02693},
  primaryClass = {cs.CV}
  year = {2019},
}

@proceedings{packnet-sfm-semisup,
  author = {Vitor Guizilini and Jie Li and Rares Ambrus and Sudeep Pillai and Adrien Gaidon},
  title = {Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances},
  booktitle = {In Proceedings of the 3rd Annual Conference on Robot Learning (CoRL)}
  month = {October},
  year = {2019},
}

@proeedings{packnet-sfm-twostream,
  author = {Rares Ambrus and Vitor Guizilini and Jie Li and Sudeep Pillai and Adrien Gaidon},
  title = {{Two Stream Networks for Self-Supervised Ego-Motion Estimation}},
  booktitle = {In Proceedings of the 3rd Annual Conference on Robot Learning (CoRL)}
  month = {October},
  year = {2019},
}

@proceedings{packnet-sfm-semguided,
  author = {Vitor Guizilini and Rui Hou and Jie Li and Rares Ambrus and Adrien Gaidon},
  title = {Semantically-Guided Representation Learning for Self-Supervised Monocular Depth},
  booktitle = {In Proceedings of the 8th International Conference on Learning Representations (ICLR)}
  month = {April},
  year = {2020},
}

@proceedings{superdepth,
  author = {Sudeep Pillai and Rares Ambrus and Adrien Gaidon},
  title = {SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation},
  booktitle = {In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)}
  month = {May},
  year = {2019},
}