Motion segmentation by adaptive mode seeking and clustering consensus

Abstract

The task of multi-body motion segmentation refers to segmenting feature trajectories

in a sequence of images according to their 3D motion affinity without

knowing the number of motions in advance. It is critical for understanding and

reconstructing a dynamic scene. This problem essentially consists of two subproblems,

segmenting features and detecting the number of motions. While the

state-of-the-art LBF algorithm achieves segmentation accuracy as high as 96.5%,

it is still disturbed by a phenomenon called over-locality. A novel mode seeking

algorithm with an adaptive distance measure is proposed to avoid this problem,

and improves the accuracy to 98.1%. The LBF algorithm is incapable of detecting

the number of motions itself. A randomized version of the mode seeking algorithm

is presented, which could detect the number as well as preserve satisfactory

segmentation accuracy. To detect the number of motions, a kernel optimization

method locates it via kernel alignment. However, it suffers from over-locality and

over-detects the number of motions. An intersection measure and two mutual

information measures are presented to solve this problem. Using these measures,

the proposed clustering consensus framework recasts the motion number detection

problem to a clustering consensus problem. It extends the kernel optimization

method from two-clustering consensus to multiple-clustering consensus. A large

number of experiments and comparisons have been done, and convincing results

are obtained.