In view of the limitation of shape matching in video sequence, i.e. when an object was moving or rotating, it took a long calculation time, a recognition method of moving objects based on contour features was proposed. First, the background image was captured and automatically updated. The contour of moving objects was abstracted by background subtraction. Second, a model of contour eigenvector was built based on the edge invariance moment and morphologic features. At last, the recognition and classification of objects were realized by analyzing and comparing the Euclidean distance between the contour eigenvector and each standard stylebook. Experimental results show that this method has high recognition accuracy, less calculation time and strong real time characteristics.