Code Monkey home page Code Monkey logo

Sensor Fusion Self-Driving Car Course

For more details , please check my blog https://blog.csdn.net/williamhyin/article/details/105159842 .

Welcome to the Sensor Fusion course for self-driving cars.

In this course we will be talking about sensor fusion, whch is the process of taking data from multiple sensors and combining it to give us a better understanding of the world around us. we will mostly be focusing on two sensors, lidar, and radar. By the end we will be fusing the data from these two sensors to track multiple cars on the road, estimating their positions and speed.

Lidar sensing gives us high resolution data by sending out thousands of laser signals. These lasers bounce off objects, returning to the sensor where we can then determine how far away objects are by timing how long it takes for the signal to return. Also we can tell a little bit about the object that was hit by measuring the intesity of the returned signal. Each laser ray is in the infrared spectrum, and is sent out at many different angles, usually in a 360 degree range. While lidar sensors gives us very high accurate models for the world around us in 3D, they are currently very expensive, upwards of $60,000 for a standard unit.

Radar data is typically very sparse and in a limited range, however it can directly tell us how fast an object is moving in a certain direction. This ability makes radars a very pratical sensor for doing things like cruise control where its important to know how fast the car infront of you is traveling. Radar sensors are also very affordable and common now of days in newer cars.

Sensor Fusion by combing lidar's high resoultion imaging with radar's ability to measure velocity of objects we can get a better understanding of the sorrounding environment than we could using one of the sensors alone.

Demo

Folder structure

  • README.md

  • media - demo image for readme file

  • ./src/

    • enviorment.cpp - main function

    • processPointClouds.h&cpp - Point-Cloud processing functions, which include filtering, segmentation, clustring, drawing bouding box ...

    • ransac3d.h&cpp - RANSAC-based segmentation of obstacles and plane

    • cluster3d.h&cpp&kdtree3d.h - KD-Tree based clustering of obstacles

    • /quiz/ - testing quiz functions for ransac and clustering

    • /render/ - rendering function for Point-Cloud

    • /Sensors/ - real word Point-Cloud data

Installation

Ubuntu

$> sudo apt install libpcl-dev
$> cd ~
$> git clone https://github.com/udacity/SFND_Lidar_Obstacle_Detection.git
$> cd SFND_Lidar_Obstacle_Detection
$> mkdir build && cd build
$> cmake ..
$> make
$> ./environment

Windows

http://www.pointclouds.org/downloads/windows.html

MAC

Install via Homebrew

  1. install homebrew
  2. update homebrew
    $> brew update
  3. add homebrew science tap
    $> brew tap brewsci/science
  4. view pcl install options
    $> brew options pcl
  5. install PCL
    $> brew install pcl

Prebuilt Binaries via Universal Installer

http://www.pointclouds.org/downloads/macosx.html
NOTE: very old version

Build from Source

PCL Source Github

PCL Mac Compilation Docs

Hao Yin's Projects

3d-object-detection-for-autonomous-vehicles icon 3d-object-detection-for-autonomous-vehicles

Project on 3D Object Detection using Lyft's level5 dataset. Obtained mAP of 0.045 on the private leader board on kaggle and ranked in the top 20% among all teams participated in the competition.

awesome-object-detection icon awesome-object-detection

Awesome Object Detection based on handong1587 github: https://handong1587.github.io/deep_learning/2015/10/09/object-detection.html

capsule_net_pytorch icon capsule_net_pytorch

Readable implementation of a Capsule Network as described in "Dynamic Routing Between Capsules" [Hinton et. al.]

copilot icon copilot

Lane and obstacle detection for active assistance during driving. Uses windowed sweep for lane detection. Combination of object tracking and YOLO for obstacles. Determines lane change, relative velocity and time to collision

cs-books icon cs-books

📚 Computer Science Books 计算机技术类书籍 PDF

darknet icon darknet

Windows and Linux version of Darknet Yolo v4 (v3/v2) Neural Networks for object detection (Tensor Cores are used)

deeplearning-500-questions icon deeplearning-500-questions

深度学习500问,以问答形式对常用的概率知识、线性代数、机器学习、深度学习、计算机视觉等热点问题进行阐述,以帮助自己及有需要的读者。 全书分为18个章节,50余万字。由于水平有限,书中不妥之处恳请广大读者批评指正。 未完待续............ 如有意合作,联系[email protected] 版权所有,违权必究 Tan 2018.06

detectron2 icon detectron2

Detectron2 is FAIR's next-generation platform for object detection and segmentation.

gen-efficientnet-pytorch icon gen-efficientnet-pytorch

Pretrained EfficientNet, EfficientNet-Lite, MixNet, MobileNetV3 / V2, MNASNet A1 and B1, FBNet, Single-Path NAS

gym icon gym

A toolkit for developing and comparing reinforcement learning algorithms.

igan icon igan

Interactive Image Generation via Generative Adversarial Networks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.