A comprehensive coarse-to-fine sports video analysis framework to infer 3D parameters of video objects with application to tennis video sequences

In this paper, we present a novel video content analysis system. An innovative 2D to 3D parameter inference algorithm is presented. It is applied to the tennis player body shape modeling, after a coarse-to-fine analysis on real world sports video sequences. As the first step, the video shots are classified in coarse level. Only shots containing appropriate body shape size are retained for the fine-level analysis. The fine-level analysis begins with a video object (VO) segmentation stage to obtain the player body shapes. The VOs then undergo training and testing stages. The training VOs are classified into serving and non-serving classes by Gaussian mixture modeling (GMM). The VOs in serving class are further clustered and the corresponding 3D parameters of a human body model are obtained manually for each cluster center. For a testing VO sequence, the VOs that contain servings are found by GMM and the initial 3D parameters are fitted to the closest matches to the cluster centers. Based on the initial guess, an innovative multidimensional optimization procedure is employed to obtain the 3D parameters. Experiments are performed on broadcasted tennis games and promising results are obtained.