yangshiyu89 / carnd-vehicle-detection-master Goto Github PK

My Vehicle Detection and Tracking project for Udacity Self Driving Car Nanodegree. This is Project 5.

In this project, the goal is to write a software pipeline to detect vehicles in a video (start with the test_video.mp4 and later implement on full project_video.mp4).

Data is provided with a labeled dataset and the job is to decide what features to extract, then train a classifier and ultimately track vehicles in a video stream. Here are links to the labeled data for vehicle and non-vehicle examples to train your classifier. These example images come from a combination of the GTI vehicle image database, the KITTI vision benchmark suite, and examples extracted from the project video itself.

Step1: Data exploration There are 8792 number of samples in cars set, 8968 number of samples in notcars set. We plot example car and not car image to have a general idea about the dataset.

Step2: Extract image features to identify car and not car content.
I used HoG (Histogram of oriented gradients), spatial_features, color histogram to extract image features.

• spatial feature (reduce the image size): spatial_features = bin_spatial(feature_image, size=spatial_size)
• color histogram feature (np.histogram on color image): hist_features = color_hist(feature_image, nbins=hist_bins)
• HoG feature (get histogram of oriented gradient after converting image to certain color space onto every block and cells, hog is in skimage.feature package): features, hog_image = hog(img, orientations=orient, pixels_per_cell=(pix_per_cell, pix_per_cell), cells_per_block=(cell_per_block, cell_per_block), transform_sqrt=True, visualise=vis, feature_vector=feature_vec)

Step3: Normalize features and use machine learning method (SVM in this task) to train dataset using optimized parameters for extracting features I used StandScaler() to normalize extracted features. And we could see before and after normalization:

In this project linear SVM is used and provided an acceptable testing accuracy. 80% of dataset has been used a training set and 20% as testing set. We shuffle data before training to avoid algortihm remembering the ordering. I tried different combination of HoG settings as well color space for feature extraction. The different option testing accuracies are included below:

Hence we select option 5 which use ColorSpace of YCrCb and Spatial_size (32,32), 32 Hist bins, orient 8, Pixels_per_cell 7, Cell_per_block of 2, HOG channel All.

Step4: Build a sliding window technique to identify car/not car content from a large image We used 6 different sizes windows to detect car or not car content. X direction cover from leftmost to rightmost, while y direction covers from near front of driving car to the horizon. We use overlay of 75% as a default. We have smaller windows mainly at the horizon for smaller car images as they will appear far away. Example detected window of car and all the sliding windows are below:

Step5: Use method like heatmap of recurring detections to avoid false positive/negative classifications As we could see there are some false positive windows which identify car image on something which are not cars. I use heatmap for the recurring detection and apply threshold to filter out low recurrence which are more likely false positive. Example of test image under heatmap is like

Step6: Build video pipeline to process streaming videos with bounding box to detect and track vehicles. I apply Advanced Lane Finding with vehicle detection and tracking together.

• Vehicle_Detection_And_Tracking_MZ.ipynb (main script)
• CameraData.p (Camera calibration data from previous project using chessboard images)
• data_carsnotcars.p (Car and not a car image data, raw images are not included due to large size, but link has been provided above in Data source section)
• ClassifierData.p, ProcessedData.p (temperaly output data files for easy resume work)
• lesson_functions.py (functions build in course which helps extract image car features)
• folder output_images (output images)
• project_video.mp4, processed_r3_MZ_project_video.mp4, processed_wLane_r4_project_video.mp4 (original mp4 and output vehicle detection and tracking without and with advanced lane finding)

As we could see from the output video, the vehicle detection is still not perfect yet as it sometimes gives pretty large bounding box onto the car which in practice means low accuracy of knowing where exactly the car is. Also there are still some false positive in some cases especially some road rails. An improvement could be using smaller sliding windows or apply higher threshold on heatmap.

Another further possible improvement is to achieve better testing accuracy using deep learning method instead of linear SVM. However using deep learning may require more carefulness in terms of responsiveness in real time driving.