alisoltani / udacitycarnd-project4 Goto Github PK

The goals / steps of this project are the following:

• Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.
• Apply a distortion correction to raw images.
• Use color transforms, gradients, etc., to create a thresholded binary image.
• Apply a perspective transform to rectify binary image ("birds-eye view").
• Detect lane pixels and fit to find the lane boundary.
• Determine the curvature of the lane and vehicle position with respect to center.
• Warp the detected lane boundaries back onto the original image.
• Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.

There are two IPython notebooks that was created during this project, the first (AdvancedLaneDetection.ipynb) contains all parts of the project except for the video streamlining. The second notebook (lanedetection.ipynb) takes code from the first notebook and makes it into an object oriented format with classes, and the video generation. I will mainly refer to the first notebook as it was written with more explanations inbetween, and only refer to the second during video generating section. There is also a lanedetection.py file that generates the video as well.

The code for this step is contained in the fourth code cell of the IPython notebook Advancedlanedetection.ipynb (also in lanedetection.py lines 12-48).

I start by preparing "object points", which will be the (x, y, z) coordinates of the chessboard corners in the world. Here I am assuming the chessboard is fixed on the (x, y) plane at z=0, such that the object points are the same for each calibration image. Thus, objp is just a replicated array of coordinates, and objpoints will be appended with a copy of it every time I successfully detect all chessboard corners in a test image. imgpoints will be appended with the (x, y) pixel position of each of the corners in the image plane with each successful chessboard detection.

I then used the output objpoints and imgpoints to compute the camera calibration and distortion coefficients using the cv2.calibrateCamera() function. I applied this distortion correction to the test image using the cv2.undistort() function and obtained the result shown in the fifth cell of Advancedlanedetection.ipynb.

For this step, I use the cv2.calibrateCamera function (that takes the image and object point generated in the previous step and apply that to cv2.undistort to generate images like: .

I used a combination of color and gradient thresholds to generate a binary image (thresholding steps at lines 50 through 93 in lanedetection.py). I use the Sobel transform (in the x dimension), the b channel in the lab for yellow and v channel in LUV for white. Here's an example of my output for this step.

First image is from the Sobel transform, second from the s channel, third from r channel, and last is the combined.

More examples are available in the Advancedlanedetection.ipynb notebook.

The code for my perspective transform includes a function called get_perspective_transform(), which appears in lines 95 through 126 in the file lanedetection.py and in the AdvancedLaneDetection.ipynb. This function generates the (src) and destination (dst) points. I chose the hardcode the source and destination points in the following manner:

    src = np.float32(
        [[572, 455],
         [0, 720],
         [1280, 720],
         [700, 455]])

    dst = np.float32(
        [[160, 0],
         [160, 720],
         [1120, 720],
         [1120, 0]])

I verified that my perspective transform was working as expected by drawing the src and dst points onto a test image and its warped counterpart to verify that the lines appear parallel in the warped image.

Once I undistort, change the image to binary and unwarp the image, the next step is to identify the lane-line pixels. For this step, I splie the image into left and right and generate to Line class objects to track each lane. This object contains the lanedetection() function that is called when the pipeline is started. Originally I had planned on using both sliding histogram technique and using known previous values, but I ended up only using the sliding histogram technique (implemented in lanedetection.py lines 178 to 244).

In this function, I first split the image and do a histogram on the bottom half, to find an appropriate starting position.

Once that is done, I start from the bottom of the image and generate rectangles, and find lane line pixels in those rectangles. I use the mean value of the pixels to update the base value position of the lanes, to be able to track curves. Once all windows are generated, I fit my lane lines with a 2nd order polynomial like this

I also tried and implemented the correlation technique, shown in the figure below, but during trials it did not perform as well as the sliding window.

Once the lane-lines are identified, we can use them as starting points for the next frame (this tehcniques result are shown below), but as mentioned before in this project new sliding window wsa used every time.

These images and code used to generate them is available in AdvancedLaneDetection.ipynb

I implemented this step in lines 338 and 380 through 390 in my code in lanedetection.py and also in the AdvancedLaneDetection.ipynb . The curvature is calculated by taking the derivative of the fitted points. The position in the vehicle is calculated in 380 to 390.

After joining the left and right images back together, here is an example of my result on a test image:

Here's a link to my video result

One issue that was faced was during shadows and changes in the road texture. This caused the s channel (in the hls transfomr) to cause false positives, which were hard to remedy. I changed the thresholds a bit and it is better, but in the video (around the 41 second mark) a slight error can be noticed. It doesn't seem to be a catastrophic error, but is an issue to focus on further ahead, but due to time restricition in this project I will leave it for a bit later. The lines are bit wobbly in the beginning, but gets better along the way.

I use an IIR filter to introduce some memory in the fitting parameters, so I get some smoothing to the lane detection. The pipleine will likely fail when the road is not flat (hills) due to the hardcoded nature of the warping function, and should be improved for those situations. Also shadows and color changes can cause the lane detection to fail and need to be worked on further.

The tracking output on the video is a placeholder and currently only identifying cases where we diverge greatly from frame to frame, it needs to be improved upon to identify actual loses in tracking.