Intelligent Automation & Soft Computing DOI:10.32604/iasc.2022.023953
Article

Moving Object Detection and Tracking Algorithm Using Hybrid Decomposition Parallel Processing

M. Gomathy Nayagam1,*, K. Ramar2, K. Venkatesh3 and S. P. Raja4

1Department of Computer Science and Engineering, Ramco Institute of Technology, Rajapalayam, 626117, India
2Department of Computer Science and Engineering, R.M.K. College of Engineering and Technology, Chennai, 601206, India
3Department of Computer Science and Engineering, Veltech Rangarajan Dr Sagunthala R&D Institute of Science and Technology, Chennai, 600062, India
4School of Computer Science and Engineering, Vellore Institute of Technology, Vellore, 632014, Tamilnadu, India
*Corresponding Author: M. Gomathy Nayagam. Email: m.g.nayagam@gmail.com
Received: 28 September 2021; Accepted: 17 December 2021

Abstract: Moving object detection, classification and tracking are more crucial and challenging task in most of the computer vision and machine vision applications such as robot navigation, human behavior analysis, traffic flow analysis and etc. However, most of object detection and tracking algorithms are not suitable for real time processing and causes slower processing speed due to the processing and analyzing of high resolution video from high-end multiple cameras. It requires more computation and storage. To address the aforementioned problem, this paper proposes a way of parallel processing of temporal frame differencing algorithm for object detection and contour tracking using the mixture of functional and domain decomposition parallel processing techniques. It has two main contributions. First, steps of frame differencing are parallelized using functional decomposition technique. Second, the processing of frames in each steps of frame differencing is again parallelized using domain decomposition technique. Finally, the performance is evaluated in Aneka Private Cloud platform and which yields to detect and track the object very swiftly and accurately.

Keywords: Temporal frame differencing based object detection; functional and domain decomposition parallel processing techniques contour tracking and cloud computing

1  Introduction

In automated video surveillance system [1], the task of detection and tracking of moving object is a important part and it can be done be different approaches. But it does not have common approaches for all applications. Background subtraction is one of the most popular and robust method for moving object detection [2,3]. But there is a challenge in background subtraction due to dynamic background, illumination changes and etc. [4]. The temporal difference method is used in [5] where they proposed reliable foreground segmentation algorithm which combines temporal image analysis with reference background image. It is suitable to adapt for background change in illumination. In [6], they extract moving object based on temporal differencing mechanism and they used Normalized Cross Correlation (NCC) technique to remove Ghost and shadow effect during temporal frame differencing mechanism. Smart cameras that perform real time video analysis are recent trends in surveillance system [7].

Due to the amount of data that needs to be processed for providing high accuracy in real time video surveillance systems, most of the algorithms are having difficult and time consuming characteristics in nature. If we perform video analysis in serial processor, it requires more computation time and uses significant amount of energy which bounds the application of such algorithm in live video analysis.

GPU are popular platform for parallel processing of objection and tracking [8] using software such as openCL and CUDA. Due to the physical constraints and high energy consumption of GPU, there is a need to modify the algorithm for the particular embedded GPU chipset. Otherwise, it results in suboptimal performance in such GPU chipsets.

Another way is the implementation of parallel algorithm on embedded systems which is based on digital signal processor (DSP) and field programmable gate arrays (FPGA) [9]. Since, this system have low number of computing units, this solution is suitable only for low video resolution and low frame rate video processing.

Cloud computing is one of the platform for implementing parallelization of moving object detection and tracking algorithm without any modification for optimization and also does not require any specialized hardware. So, implementing parallelization of moving object detection and tracking algorithm in cloud platform is more efficient and cost effective [10–12].

In this paper, parallel version of temporal frame difference based moving object detection with contour tracking is presented. In order to achieve functional decomposition, there are three master threads are created in this proposed work. First one is responsible for motion detection using temporal frame differencing, second one is responsible for noise removal in foreground image and third one is responsible for blob detection and track the individual objects. The moving object detection thread is again divided into child thread for inherent parallelization of pixel by pixel subtraction operation. The work will outperforms to detect multiple objects at same time with high speed with good accuracy.

Remaining of the paper is organized as follows: Section 2 surveys the relevant work done using cloud computing and other technologies. Section 3 describes the proposed work. Section 4 describes results and performance evaluation. Section 5 discusses conclusion and future work.

2  Proposed Work

The proposed cloud computing based rapid object detection and tracking system is elucidated in this section.

2.1 Overview of the Work

The main aim of this article is to present cloud computing based rapid object detection and tracking system. Most of the object detection and tracking algorithm requires more computational and storage for processing video. The cloud computing will provide environment which has more computational and storage requirements to process the video for detecting and tracking object in it with very rapidly. The following Fig. 1 shows the architecture of cloud based moving object detection and tracking system in video. The proposed system assumes the camera is fixed in the particular surveillance area with fairly wide angle of view.

Figure 1: Architecture of cloud based moving object detection and tracking system

The frequently captured video file is transferred to cloud environment through GUI and cloud manager where video are sliced into frames and each frames are considered as a work unit of cloud environment. Each of the work unit is distributed to executor by using cloud scheduler. The system components of proposed system architecture includes: video capture, motion detection, Image Filtering, Blob detection, contour detection, updatation and tracking.

In order to parallelize these components, the proposed system architecture uses functional decomposition based parallelism. Hence, each functional component of temporal object detection and tracking algorithms are executed parallelly by the worker node. Again, these functional decomposed components are internally parallelized using inherent sequential parallelizing mechanism which is executed by the individual workers nodes. The Fig. 2 shows the functional decomposition based parallelizing mechanism to execute the functional components of object detection and tracking algorithm parallely.

Figure 2: Functional decomposition based parallel execution for temporal frame differencing

2.2 Video Capture

The input can be taken in two ways using camera or giving path for stored video file which are captured from camera sensors. Live video streams need to be processed. Once the video captured which are converted in to frames and processed. Frames in videos are sequence of images which is a representation of two dimensional images as a finite set of digital values called pixel [13–15] in digital computer. Digital images are represented as 2D matrices which contains finite set of elements px, y which is the color information for the corresponding pixel in the image at the given location x, y.

2.3 Motion Detection Using Temporal Frame Differencing

In this work, detection of moving object using temporal frames differencing [16] which is the simplest background subtraction techniques. Also, it is a non-recursive method; hence there is no need to maintain the history of frames in buffer. So, memory requirement for this algorithm is low compared to other existing algorithms. In this method, current frame’s pixel values are subtracted with previous frames pixel value. Based on the thresholding value, moving region of the image is extracted and identifies the both foreground and background [17]. The resultant of absolute pixel value is zero indicates, there is no moving object or non-zero value of this pixel value indicates there was a moving object in frames.

According to the method describe above, the difference between the frame at t and frame at t−1 is determined as follows [18,19]:

Dk=|Ix,y,t−Ix,y,t−1| (1)

For adopting the changes in background illumination [19,20], the reference image is updated at each iteration as follows:

Bt+1=(1−α)Bt+αIx,y,t (2)

The pixel value of frames/image is represented as a matrix and matrix subtraction operations carried out for motion detection. The size of image is divided into smaller kernel like N × N matrix and each kernel of adjacent frame pixels are created as a thread which are executed in different cloud executor using inherently sequential decomposition mechanism. The Fig. 3 shows the inherently sequential based functional domain decomposition for temporal frame differencing.

Figure 3: Inherently sequential based functional domain decomposition for temporal frame differencing

2.4 Noise Removal

Some of the background frames have spurious movement of object. It leads to mistaken in detect and track the object. So, this noise causing elements have to be filtered. For this purpose, this work used two basic morphological operations viz erosion and dilation. The basic effect of dilation mathematical morphological operation is to gradually increase the image boundary of region of foreground pixels whereas erosion mathematical morphological operation gradually shrinks the background [20]. Hence, it removes the noises.

The erosion of binary image f by a structuring element s is denoted by fθs which produces g=fθs. It reduces the size of regions of interest.

If g(x,y)={1,′s′fits′f′0,otherwise

The dilation of an image f by a structuring element s is denoted by f ⊕ s which produces g = f ⊕ s. It adds a layer of pixels to both inner and outer boundaries of region.

If g(x,y)={1,′s′hits′f′0,otherwise

The number of iteration for each erode operation is depends on how much noise presents in the video frame. So, here we applied dilation operation applied on the output of the erosion operation. These operations remove noise, isolation of individual elements and distinct element in an image and finding intensity holes in an image. Both operations take two pieces of data as input. One is the image which is to be dilated and another one is set of coordinate points known as structuring elements/kernel with the size of N × N. Each N × N size of frames is processed by different worker nodes of cloud environment using functional domain decomposition approach.

2.5 Blob Detection

It is a mathematical step to detect specific region based on some property in digital image. Blob made up of pixels having specific property in common and pixels which are combined together if they share common neighborhood pixels that have values greater than the threshold are the property of interest. After absolute differencing and noise removal applied in each video frames, the output of the pixel value is greater than binary threshold which is eligible to form a blob. All the adjacent neighbor of each such pixels are tested for eligibility and, if they satisfy, all those pixels are combined together to form blobs. In order to blob created by spurious movement, which are considerably in small, the size of detecting blob is limited by choosing blob threshold. By default, in this work the threshold value for blob detection is 15 and also applied smooth Gaussian filter with small kernel size (i.e., 3 × 3). Functionally decomposed kernels are processed by different worker node as individual thread. The output of this module is given as the feedback to detection system for learning.

2.6 Contour Detection and Tracking

Contour is the boundary of object pixel above the threshold value. It is edge based features which are insensitive to illumination change. Contour analysis output uses a linked list which consists of detected blobs. All the promising contours are found and reconstructed a full hierarchy of nested contour by using RETR_TREE Mode. The end points which are necessary for drawing the contour are extracted by using CHAIN_APPROX_SIMPLE method. In this work, once contour is identified, each contour is tracked separately with individual thread of different worker nodes in cloud environment. Fig. 4 shows the sample video frames and corresponding detect contour in the frame.

Figure 4: Sample capture video frame and its corresponding contour detected frames

The proposed algorithm is shown in the Fig. 5.

Figure 5: Pseudo code of proposed algorithm

The following Fig. 6 shows the temporal difference between contiguous frames and extracted foreground object using threshold, detected foreground objects.

Figure 6: Temporal frame difference, with threshold output and detected foreground in contiguous frame

3  Performance Evaluation

We have run some tests in order to compare the performance of different execution. We set the different tests by varying the following parameters:

•   Number of Video Frames

•   Frame rate

•   Video file size

•   Video Resolution

•   Kernel Dimension for processing

The performance of the proposed approach is evaluated in local standalone machine with Intel® Core™ i5-3380M CPU@ 2.90 Ghz with 4 GB RAM, GPU Node( NIVIDIA GeForce210 with 4 GB Ram) and Aneka Cloud composed of 3 nodes–1 manager and 2 executor where each arehaving the configuration of Intel Core i5 processor systems with 8 GB RAM. The entiremachines in the test were running in Windows 7 professional SP1 and .Net Framework 2.0 with EMGUCV. With the intention of checking the proposed approach, it is tested over several videos which are chosen from the CAVIAR and VISIOR data set. The accuracy of object detection is shown in Tab. 1 and also Tabs. 2–7 shows the total elapsed time of object detection and tracking in various video with varying kernel dimension for processing without multi-threading in standalone i5 processor, with multi-threading in standalone i5 processor, without multi-threaded GPU node, with multi-threaded GPU node, Aneka Cloud with single executor and Aneka Cloud with 2 Executor.

The data represented in the Tab. 1 show that object detection accuracy of various video data sets with different resolution which is calculated by the following formula.

ar=Tp+TnTp+Tn+Fp+Fn×100 (3)

where Tp, Tn are the true positive and true negative rates. Fp, Fn are the false positive and false negative rates. From the aforementioned table, it is observed that we get high accuracy of object detection from high resolution video. But the processing time of high resolution video is high.

MSE is one of the measurements for identifying similarity between frames. If we are using MSE is only the measurement of identifying similarity between frames, it runs into problem. Hence in this work, we used Structural similarity index developed in [21].

SSIM(x,y)=(2μxμy+c1)+(2σxy+c2)(μx2+μy2+c1)(σx2+σy2+c2) (4)

SSIM attempts to model the perceived change in the structural information of the image. It compares two window rather than compare entire image in MSE. SSIM value can vary between −1 to 1 where 1 indicates perfect similarity. The Fig. 7 shows the SSIM analysis between each frame. Here, SSIM is varied from 0.825 to 1. This results shown that, there is a change of luminance and/or contrast. But it will not the affect detection accuracy of foreground using temporal frame differencing algorithm.

Figure 7: Frame wise SSIM analysis

The efficiency of the proposed approach is tested in terms of total elapsed time of object detection and tracking process and speedup ratio between Standalone system vs. GPU node vs. Aneka Cloud platform with and without multi-threading by varying kernel size of video frames are given in Tab. 2.

Fig. 8 shows histogram of processing time for object detection and tracking. It shown that object detection and tracking in private cloud environment (i.e., Aneka with 2 executor node) will outperform other parallel processing technique with hybrid decomposition technique.

Figure 8: Processing time histogram of object detection and tracking

Figs. 9 and 10 shows the results of speedup ratio between standalone node vs. GPU node vs. Aneka Cloud single and 2 Executors with and without multi-threading. From the results, it is observed that Aneka cloud platform always gives better performance even though which is having communication overhead between nodes for detecting and tracking the objects in video without compromising detection accuracy and also without investing any specialized hardware platform.

Figure 9: Speedup ratio between standalone node vs. GPU node vs. Aneka Cloud with single executor without multi-threading

Figure 10: Speedup ratio between standalone node vs. GPU node vs. Aneka Cloud with 2 executor with multi-threading

4  Conclusion and Future Work

This work proposed a cloud computing based rapid object detection and tracking algorithm for video solution applied with temporal frame differencing and contour matching techniques using functional decomposition technique. In this work, each functional component of temporal object detection and tracking algorithms are executed parallelly by the worker node. Again, these functional decomposed components are internally parallelized using inherent sequential parallelizing mechanism which is executed by the individual workers nodes of Aneka cloud environment. The performance of proposed approach is compared and evaluated with standalone machine, GPU node and Aneka Cloud environment. The performance of cloud computing based object detection and tracking outperforms well compare to other platform without investing any specialized hardware architecture. In future, this work is plan to extend by extracting the various features (like LBP, HOG) of detected blobs and improve the accuracy of detection and tracking of blobs and improve the computation as a service for very swift object detection and tracking algorithm through the design of application level task scheduler for cloud environment. This work more suitable for automation field. We plan to extend this work in real time both single and multiple camera whereas currently we used to test on only stored videos. Also, design the storage cloud architecture for video processing and surveillance application.

Acknowledgement: I am extremely thankful to Management, Principal, Head of the department of Computer Science and Engineering, Ramco Institute of Technology, Rajaplayam for providing me laboratory facility in RIT-AI Research Centre to carry out this work in successful way.

Funding Statement: The authors received no specific funding for this study.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.