This post will detail my plans and progress for an autonomous car. I’ll be using what I learned from
building the balancing robot and courses though Udacity and MITx.
The goal is to take a small 1/10
remote controlled car and have it navigate a course on its own. To navigate the course, I plan to use an
Android phone, GPS, and an accelerometer/gyroscope. I will record a video stream through the phone
and then process it on my laptop using Python and the computer vision library OpenCV.
I chose a 1/10 scale car so that it would be large enough to hold all of the electronics. I was worried that the cheaper and more common 1/16 scale cars would be too small. I found this car on Amazon for about $100. Here are some pictures of it below. I've already started working on the electronics.
Below is a block diagram I drew that shows my preliminary design for the software portion of the car.
At the same time, heading data from the GPS and gyroscope are passed into another Kalman filter to get an accurate heading on the car. The position and heading data from the Kalman filters is then compared to a GPS waypoint given to the sensor and motor control. Next, a heading toward the desired GPS waypoint is calculated, and the appropriate signal is sent to the steering servo in order to point the car in that direction.
While this process is occurring, the camera from the Android phone will stream images of the car’s path back to my computer. On the computer, I will use Python and OpenCV to process the images. The images will be processed looking for the edges of the road (width) and also looking for obstacles in from of the car. Data from the OpenCV program will be sent back to the motor control, allowing the car to deviate from its previously calculated heading and avoid objects that may get in the way.
Below is the preliminary work I’ve done on the image processing. I’m using an app called Barnacle to set up a local network between my phone and laptop and then sending the camera feed over HTTP. Then I’m using Python to grab a snapshot of the camera feed. I’ll walk through the steps of the image processing below. I’ll save the discussion of the theory and mathematics behind these processing algorithms for another time and just stick with the results for now.
Here’s the original image on the left. The first step is to convert it to grayscale, shown on the right.
From the gray image I run a Gaussian filter to reduce the noise and avoid finding false lines later on in the processing (not shown). Then the result of the Gaussian filter is passed into a Canny edge detector to locate the edges.
From the Canny image, you can already see the outline of the road. But if you look closely at the Canny image, you’ll notice that those aren’t quite continuous lines. There are spots where the line breaks, which will make doing any useful mathematics on it later more difficult. To fix this issue and clean up the image further, I’m applying a Hough transform to the Canny output. This transform is fairly complicated, but the quick summary is that it looks for sections in the image that aren’t continuous but that form part of a larger line. For this part, I’m also giving it a theta angle to restrict which lines it returns.
That’s it for now. From here I can make sure that the car stays in the middle of the road.
Next plans are to add obstacle detection.