Multiple motion segmentation with level sets

Motion segmentation of an image sequence belongs to the most difficult and important problems in video processing and compression, and in computer vision. In this paper, we consider the problem of segmenting an image into multiple regions possibly undergoing different motions. To this end we use level sets of functions evolving according to certain partial differential equations. Contrary to numerous other motion segmentation algorithms based on level sets, we compute accurate motion boundaries without relying on intensity boundaries as an accessory. This will be illustrated on examples where intensity boundaries are hardly visible and yet motion boundaries are accurately identified. The main benefit of the level set representation is in its ability to handle variations in the topology of the level sets. As a result, it is only necessary to know the total number of distinct motion classes and their parameters. We describe an automatic initialization procedure that is based on feature point correspondences and K-means clustering in a 6-parameter space of affine parameters. We illustrate the performance of the proposed algorithm on real images with both real and synthetic motion.