Multi-Target Tracking of Time-Varying Spatial Patterns [Paper]
Abstract
Time-varying spatial patterns are common, but few computational tools exist for discovering and tracking multiple, sometimes overlapping, spatial structures of targets. We propose a multi-target tracking framework that takes advantage of spatial patterns inside the targets even though the number, the form and the regularity of such patterns vary with time. RANSAC-based model fitting algorithms are developed to automatically recognize (or dismiss) (il)legitimate patterns. Patterns are represented using a mixture of Markov Random Fields (MRF) with constraints (local and global) and preferences encoded into pairwise potential functions. To handle pattern variations continuously, we introduce a posterior probability for each spatial pattern modeled as a Bernoulli distribution. Tracking is achieved by inferring the optimal state configurations of the targets using belief propagation on a mixture of MRFs. We have evaluated our formulation on real video data with multiple targets containing time-varying lattice patterns and/or reflection symmetry patterns. Experimental results of our proposed algorithm show superior tracking performance over existing methods..
Results
Efficient Mean Shift Belief Propagation for Vision Tracking [Paper]
Abstract
A mechanism for efficient mean-shift belief propagation (MSBP) is introduced. The novelty of our work is to use mean-shift to perform nonparametric mode-seeking on belief surfaces generated within the belief propagation framework. Belief Propagation (BP) is a powerful solution for
performing inference in graphical models. However, there is a quadratic increase in the cost of computation with respect to the size of the hidden variable space. While the recently proposed nonparametric belief propagation (NBP) has better performance in terms of speed, even for continuous hidden variable spaces, computation is still slow due to the particle filter sampling process. Our MSBP method only needs to compute a local grid of samples of the belief surface during each iteration. This approach needs a significantly smaller number of samples than NBP, reducing computation time, yet it also yields more accurate and stable solutions. The efficiency and robustness of MSBP is compared against other variants of BP on applications in multi-target tracking and 2D articulated body tracking.
Results
Dynamic Near-regular Texture Tracking and Manipulation [Paper]
Abstract
A near-regular texture (NRT) is a geometric and photometric deformation from its regular origin -- a congruent wallpaper pattern formed by 2D translations of a single tile. A dynamic NRT is an NRT under motion. Correspondingly, the basic unit of a dynamic NRT is a well-defined texton, as a geometrically and photometrically deformed tile, moving through a 3D spatiotemporal space. Although NRTs are pervasive in man-made and natural environments, effective computational algorithms for NRTs are few. Through a systematic and quantitative comparison study of multiple texture synthesis algorithms, we are able to show that faithful NRT synthesis has challenged most of the state of the art texture synthesis algorithms. Our recent work on static NRTs analysis and manipulation (SIGGRAPH 2004) is the first algorithmic treatment aimed specifically to preserve the regularity and randomness in real world near regular textures.
The theme of this project is to address computational issues in modeling, tracking and manipulating dynamic NRTs. One basic observation on dynamic NRT is its topology invariance property: the lattice structure of a dynamic NRT remains invariant despite its drastic geometry or appearance variations. We propose a lattice-based Markov-Random-Field (MRF) model for dynamic NRT in a 3D spatiotemporal space. Our dynamic NRT model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations. Our model behaves like a network of statistically varied springs. Based on our dynamic NRT model, we develop a tracking algorithm that can effectively handle the special challenges of dynamic NRT tracking, including: ambiguous correspondences, occlusions, illumination variations, and appearance variations. Furthermore, we implement a dynamic NRT manipulation system that can replace and superimpose augmented images on a dynamic NRT from an unknown environment.
Tracking Results
Texture Replacement
Related Publications
Jingchen Liu and Yanxi Liu. Multi-Target Tracking of Time-Varying Spatial Patterns, CVPR 2010. [PDF]
