How and why I built TraceItOut - A video summarizer
About me
I am a software development engineer who is currently working at Mentor Graphics, India. I have studied Computer Engineering from Thapar Institute of Engineering & Technology.
The problem I wanted to solve
In today’s scenario crime rates are increasing significantly. But along with the increase in crime rates, we also have an increase in technological advancements, which are also increasing by leaps and bounds. These technological advancements are majorly in the field of artificial intelligence, cloud computing, and other fields of computer science and technology. The basic concept is that if the crime is increasing, technology is also increasing simultaneously and in a much better place. These technological advancements, if worked upon the incorrect path, can lead us to better tackling the crime, thus crime rates can be significantly reduced.
Let’s take a particular scenario into consideration. A cycle of a student gets stolen during a certain period of time. When the student wishes to check the criminal activity by the surveillance camera installed in the site of the crime, he/she has to watch the entire video, for example of a full week, in order to identify the crime. Now, this is not at all a simple and easy procedure from the victim’s point of view. He/she is only interested in that particular frames of video where the cycle was stolen, not the complete week video, which indeed is completely irrelevant to him/her. In order to solve this particular problem we will be using Trace It Out, which will not only save victim’s time by providing him/her with only that section of the video which might be of his/her usage but also save money of the University administration and thus will be social and economically beneficial. TraceItOut can thus prove out to be a revolution in the field of theft detection.
What is TraceItOut - A video summarizer ?
We have created a software application that is directly deployable to a computer, raspberry pie, and mobile app. The application is capable of recording the video, real-time analysis of video, object detection, and motion detection. Object Detection is used to fast search of the object inside the video if required.
Tech stack
Below is the list of technologies and tools used to develop the application?
- Python: I have selected python as a language to develop the application because we need some machine learning and computer vision library which are easily available in python.
- Machine Learning: The project requirement is to detect an object for the fast searching of an object inside the videos. So I need to train different machine learning models.
- Computer Vision: I have to use background subtraction to detect motion in the video.
- Raspberry Pie: I used raspberry to deploy the application to create a standalone product.
The process of building TraceItOut - A video summarizer
- Motion Detection:
Real-time segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated visual surveillance, human-machine interface, and very low-bandwidth telecommunications. A typical method is background subtraction. Many background models have been introduced to deal with different problems Background subtraction involves calculating a reference image, subtracting each new frame from this image, and thresholding the result. What results is a binary segmentation of the image which highlights regions of non-stationary objects.
Class Diagram
- Object Detection:
Recent advances in object detection are driven by the success of region proposal methods and region-based convolutional neural networks (RCNNs). Although region-based CNNs were computationally expensive as originally developed in, their cost has been drastically reduced thanks to sharing convolutions across proposals. The latest incarnation, Fast R-CNN, achieves near real-time rates using very deep networks when ignoring the time spent on region proposals. Now, proposals are the test time computational bottleneck in state-of-the-art detection systems.
Our object detection system, called Faster R-CNN, is composed of two modules. The first module is a deep fully convolutional network that proposes regions, and the second module is the Fast R-CNN detector that uses the proposed regions. The entire system is a single, unified network for object detection. Using the recently popular terminology of neural networks with ‘attention’ mechanisms, the RPN module tells the Fast R-CNN module where to look. Introduce the designs and properties of the network for region proposal. We will develop algorithms for training both modules with features shared. A Region Proposal Network (RPN) takes an image (of any size) as input and outputs a set of rectangular object proposals, each with an objectless score. We model this process with a fully convolutional network, which we describe in this section. Because our ultimate goal is to share computation with a Fast R-CNN object detection network, we assume that both nets share a common set of convolutional layers. To generate region proposals, we slide a small network over the convolutional feature map output by the last shared convolutional layer. This small network takes as input an n × n spatial window of the input convolutional feature map.
Snapshot of Working Product
- Trace It Out Algorithm:
In order to implement the reduction in the video size and thus the implementation of the object detection, a self-made algorithm, which we call Trace It Out algorithm can be used. The entire video after motion detection is divided into chunks of specific time (eg. 1 minute each). Now from this video of I minute, objects are detected and will be pushed to a text file. This file will be having multiple sets of objects. I set of objects corresponding to each time period. When the user enters the object tag, that time period where the object is found will be returned.
Final Numbers
Challenges I faced
- Learning new technology like a raspberry pie to create a standalone product. It was my first time interacting with hardware.
- To develop the code for the final searching object in the algorithm. I tried a lot of trial and error methods to develop the final algorithm.
Key learnings
- Don’t give up: whenever you are the stage of giving up like you are not able to develop this feature keep working hard in that direction. Finally, you are able to implement what is required.
- Don’t follow a specific place to learn about single technology always use multiple sources.
- It’s always a good practice to develop the project in an iterative way.
