Introduction


Say it's date night. You don your tuxedo, buy some flowers, pick your date up, and drive to your destination only to spend an hour to find a decent parking spot. Isn't it frustrating? As of 2011, there were more than 1 Billion vehicles on road around the world, and it has been almost 9 years since then. There's a good chance that the number will double in the coming year. But do we have enough infrastructure to cater to such a large number of automobiles? Well, that's a subject for another discussion.


Coming back to our immediate concern of parking, Americans spend an average of 17 hours per year searching for parking. As they say, time is money, this wastage of 17 hours results in a cost of $345 per driver in wasted time, fuel and emissions. To put things in perspective, collectively, Americans spend $73 Billion a year searching for parking spots.


If you are someone living in the UK, things are even worse for you. British drivers waste an average total of nearly 4 days every year looking for parking spaces. So the question then becomes: "Would we continue to waste hours in search of parking for the rest of our lives? Not really!


What's the solution? How about a Parking Spot detection system powered by Convolutional Neural Networks? Let's try and understand the approach that can be used to solve the 'Parking Problem' using deep learning.


Problem Statement


Given an image captured from camera mounted on wall for capturing top view image of parking lot, identify occupancy status of parking slots in given parking area.


Solution


The setup's quite simple. All we need is a wall-mounted camera that can capture a top view image of the parking lot to be assessed. The problem at hand can be solved in 2 ways.
Firstly we might manually mark parking slots in image obtained from camera. This way we can crop region of manually created parking lot and pass it to machine learning model for predicting whether patch of image contains a vehicle i.e occupancy status as busy or not occupied i.e occupancy status as free.
Secondly we can perform transfer learning on pretrained object detection model for identifying possible parking locations in given parking image and also predict occupancy status.
Let’s dive in for understanding how we solved problem using both approaches.


Solution Approach 1 :


First, we might manually mark parking slots in the image obtained from the camera. This way we can crop region of the manually created parking lot and pass it to a machine learning model for predicting whether the image contains a vehicle i.e occupancy status as busy or not occupied i.e occupancy status as free.


Training workflow :


  • Collect parking images from camera stream.
  • Manually identify parking patches in image specific for particular camera.
  • Extract patches from parking images and resize & normalize.
  • Train CNN using patches of parking lot and their occupancy status.
  • Save model for future use.

Workflow of using model :


  • Extract image as a sequence of frames from camera stream.
  • Crop out patch region from image
  • Resize and normalize it.
  • Pass it to pretrained CNN and predict occupancy status as busy/free.



Patch 1


Patch 2


Sample parking images :



Parking images with predictions :


Predictions made on parking images. Green bounding box represents free slot and red represents occupancy status as busy.



Solution Approach 2 :


In this approach, we can perform transfer learning on pre-trained object detection model for identifying possible parking locations in a given parking image and also predict occupancy status. Transfer learning can be performed using faster RCNN. It's pre-trained on Imagenet dataset. The model has been trained using luminoth. The workflow can be simplified as follows:


Training workflow :


  • Collect parking images from camera stream.
  • Manually annotate parking slots in image using labelImg tool.
  • Convert xml generated to csv using utility scripts from tensorflow object detection repository.
  • Train Faster -RCNN model using luminoth on training set and evaluate on validation set.
  • Save trained model for inference on test dataset.

Workflow of using model :


  • Extract image as a sequence of frames from camera stream using OpenCV.
  • Pass it to Faster RCNN model for predicting parking locations and their occupancy status.

Sample parking images :



Parking images with predictions :


Red bounding box represents free and blue bounding box represents busy.



Results:


  • The trained model obtained an accuracy of 96% with f1 score of 0.95 by training CNN from the ground up. The possibilities that can be unlocked using such products are limitless.


Use Cases :


  • Curbing the volume of automobiles sold is not the solution. Effective management, on the other hand, can be the golden key to all our problems.
  • The model can be deployed on smart cameras for predicting parking locations on real-time feed. The model can be also deployed on a backend server and integrate with android app for predicting free parking location in real-time. The real-time feed can also be used to smartly monetizing the parking spots. Through the app, you might book a parking spot in advance and be carefree. As said, the possibilities are limitless.
  • Artificial Intelligence won't really take over the world and hunt us as most sci-fiction work portrays. But it would definitely make our lives much simpler and easier.


References :