Activity Recognition from Infrared Surveillance Videos
Security is an emerging paradigm which has achieved predominant imporatnce in every field. The key objective of human security is to prevent the occurence of risks or threats and focus on early action. One such area that requires prior action in order to ward off loss with respect to both life and property is surveillance monitoring. With the current rate of growth in the amount of security feed being generated, the need to continuously monitor the feed has become a pressing concern. Petabytes of footage is being generated each day and it has become very difficult to monitor all the generated security feed. So, the need for ways to efficiently monitor the video feed without human intervention arises and human activity recognition solves that problem.
Problem Statement
Activity recognition has become an active research area over the past decade owing to the number of applications it has, ranging from healthcare to security, smart surveillance to intelligent environments and human computer interactions. Recognition of human activities is a challenging problem in itself owing to the varied activities a person can perform and also the varied ways in which a single activity can be done. Many approaches have been proposed with a view to solve the task at hand, but very limited seem to focus on infrared spectrum. Therefore, the need for an efficient recognising system in infrared scenarios persists and I had been working on developing an AI-based system that can stand as a solution for the same.
Proposed solution
With a view to provide an efficient solution to this problem of activity recognition, an AI-based system has been developed that focuses on the recognition of human activities especially from IR surveillance videos. The primary goal is to correctly classify human actions that take place in infrared specturm from surveillance cameras. For automating the task of surveillance inspection, various techniques of Deep Learning have been utilized, to perform the tasks of detecting and tracking humans and also recognising the activities they perform. The outcome of the project is a developed activity classification system for IR surveillance scenarios. The system contains modules for detecting humans and classifying their activities in IR scenarios. The system can also be found useful in determining if the detected humans are trying to pose a threat to the environment by checking if they are carrying any sort of guns.
Requirement Analysis
Various businesses and organizations set up surveillance systems for their safety and security. Many smart surveillance systems are introduced into the current market regularly owing to the huge demand for surveillance and monitoring of footage. This is where this project can grab an opportunity to provide a solution with respect to recognizing human activities. This solution focuses on detecting different activities in infrared scenarios. Enabling a feature that can recognize human activities has its own benefits. It can recognize unusual activities and can deter threats which might have occurred otherwise. It reduces the human effort required to continuously monitor the CCTV footage, automates the process of monitoring and brings down the need for hiring a large number of people for the same task.
Solution in Brief
The project is a unified system of different modules namely, Human Detection, Firearm Detection, Activity Classification and Human Tracking. The implementation of these modules have be done using various Deep Learning and Computer Vision techniques. A lot of experimentation had to done to decide upon the optimal parameters that would give the best performance.
The above mentioned modules have been integrated to form a unified, robust and efficient activity recognition system that can be used as a smart surveillance solution. The system achieves good performance in recognising human actions with low latency. The action classes the system was trained upon include general human actions as shown below:
Experience and Challenges faced
Although this project has been completed in a span of 4 months, there were many challenges that had to be faced. Working with huge amounts of data required lot of time, especially during manual tasks like preparation and pre-processing for classification task.
Other challenges faced with respect to data include:
- Resolution of the night-vision cameras
- Ambiguous postures for classification
- Partial visibility or other environmental factors deteriorating the image quality
Conclusion
The system has been automated to identify human actions and the presence of firearms using Deep-learning techniques. Since less work has been done in the area of action recognition in infrared surveillance, this system stands as an efficient solution which can be relied upon. The system has been trained on limited action classes so far. The classes can be extended according to the requirements and can be used by organisations where security is an utmost concern.