Tracking multiple moving objects in moving camera images

Master thesis in the field of electrical engineering (communication-system)

Abstract

Tracking multiple moving objects in moving camera images

Target tracking is one of the most important issues in machine vision science. The problem of tracking multiple targets in images obtained from a moving camera has been evaluated in this thesis. To solve this problem, two general methods are presented in this thesis. The first method is mainly based on block matching algorithms. In this method, by using motion compensation and categorizing different motions in the image, the target and background areas will be determined. Then the tracking operation has been done using the Kalman filter. According to the obtained results, this method has the ability to track targets with high accuracy in most situations. The second proposed method is basically based on the well-known point matching algorithm called SIFT. In this method, by making a match between the key points of the target area and the corresponding points in the next frame, an attempt is made to find the target area in successive frames. The problem of tracking the target areas in the frame sequence is done by Kalman filter. Due to the fact that the target is checked locally in this method, this method has great ability in different tracking situations such as fading. According to the graphs and tables, it can be concluded that this method has a much stronger performance than most existing methods in terms of accuracy in many cases of tracking. 

1-1- Introduction

Today, scientific advances have deeply affected human life. At the same time, with the introduction of technology into the personal life of society, the presence of devices and tools that play the role of the interface between man and machine is felt more and more day by day. An example of these devices are cameras[1]. The widespread use of these tools in today's societies, especially in more industrialized countries, is undeniable. Therefore, today, improving the quality and facilities of cameras is considered as an important factor in increasing their efficiency. One of the most important branches of science that investigates these issues is called machine vision[2].

One of the main goals of machine vision is to make cameras intelligent in order to use them in surveillance systems[3], commercial, military and other applications. For this purpose, extensive studies have been conducted in order to create new methods of intelligentization and also to improve existing methods. Most of these studies are focused on the detection [4] and tracking [5] of targets [6]. The general purpose of conducting such studies is to reduce the volume of calculations and increase the accuracy in the stages of detection and tracking. In general, target detection means detecting a region of the image that can be considered as a candidate [7] for the target region. For example: specifying the areas of the image that are related to the license plate, or also revealing the areas of the image that can be considered as areas related to the human face. Also, the purpose of tracking is to specify the desired area in the set of consecutive frames. In this way, the general path of the target will be determined in a sequence of time during successive frames. In the following discussion, we will briefly introduce various tracking systems and their components as well as their operation. 1-1-1 Structure of tracking systems Different tracking systems are divided into different categories based on their applications. Cameras and targets are the main components of such systems. Therefore, just as these components play a decisive role in the type of tracking systems, they are also very important in determining the type of methods used in these systems. The systems have significant differences based on the number, type, and other conditions of the cameras and objectives. Likewise, these differences can be seen in the tracking methods used in them. In the following, we will discuss the factors that are effective in creating these changes.

1-1-1-1-          Camera

As the main component of the tracking system, the camera has the task of creating a sequence of frames over time.The type of cameras used, the number and the way they are placed will play a big role in determining the appearance of the frames. This effect is sometimes to such an extent that it creates methods with a different foundation.

For example, the tracking methods in systems with visible cameras [8] are completely different from the methods used in systems with infrared cameras [9]. This problem comes from the fact that infrared cameras have a kind of prior information [10] about the targets. This means that in the images obtained from these cameras, the targets have stronger color intensity [11] than their surroundings. As a result, they have more detection capabilities. Although some of the presented algorithms can be applied to the images obtained from both visible and infrared cameras, the efficiency of these algorithms in these images is significantly different.

In addition, the number of cameras used is also one of the most important factors in determining the method used in tracking. The difference in the viewing angle of the cameras creates different images from different angles of a certain scene. In this situation, it is necessary to find the corresponding points in the frames obtained from all cameras and also to calibrate [12] the cameras. It can be seen that these methods are generally different from tracking methods based on a single camera.

In addition to the above, camera motion[13] should also be considered in some cases. This means that sometimes, in addition to the targets, the camera also has movement. In these cases, the components in the consecutive frames move relative to each other. This is despite the fact that some of these movements are due to the movement of the camera and some are also due to the presence of movement in objects. Therefore, the ultimate goal is to distinguish between the movements that are caused by the camera and the movements that are real. The need to perform this action is one of the main things that is not considered in tracking targets in fixed camera systems. The type of targets, their number as well as their appearance change in the sequence of frames are the determining factors in choosing the appropriate tracking method for the system. According to these differences in the structure of targets, different methods have been created.

In some cases of tracking, the target has special appearance characteristics. For example, sometimes the tracking of the human face is considered. The human face has a certain dominant[14] consisting of eyes, nose and lips in a pink oval shape. In detecting these cases, the algorithm only looks for areas of the image that have the ability to match the desired dominant for the human face. In this way, as it can be seen, the presented method is specific to tracking in this case and is different from other tracking methods in general.  

The number of targets in the image is also one of the important and determining factors in the tracking algorithms. With the increase in the number of targets, new and major problems arise in the fields of detection and tracking. One of the most important of these problems is the fading of targets [15] and the placement of targets in a similar position [16]. Algorithms presented by focusing on these problems have provided ways to solve them.

Change of appearance [17] of the target is one of the major problems related to its appearance conditions. This means that sometimes goals do not have a rigid structure[18]. Therefore, they look different in successive frames. For example, the appearance of a pedestrian undergoes various appearance changes in successive frames. These changes are not seen in the appearance of solid objects such as cars. Therefore, the proposed methods should be such that they have the ability to adapt to these changes in the appearance conditions of the targets. 1-1-2 How tracking systems work Although it was mentioned in the previous articles that changes in the structure of tracking systems can lead to major changes in the tracking methods used, these methods have many commonalities in many aspects and basic principles. The main difference between these methods is in how to implement the general steps. In this thesis, we generally divide the algorithms used in target tracking systems into two main categories based on the use of predictability[19].  In the following, we will introduce each of these two categories and state their advantages and disadvantages.