In this paper, a semi fragile video watermarking scheme based on chromatic DCT is proposed, which can protect copyright and tamper detection simultaneously. Firstly, through the experiment, it is found that the chroma block has a more stable prediction mode relative to the luminance block, which can effectively reduce the asynchronous probability brought by the change of the prediction mode. At the same time, blocks contains more the number of nonzero (NNZ) coefficients, the less the prediction mode changes. Therefore, for each macroblock, the algorithm sort its 4 x 4 sub macroblock according to the NNZ of chroma DCT coefficients, then select the sub block which has most NNZ residual coefficients. According to the secret key K and medium frequency stability of Intra 4 x 4 coefficients, modulate the relationship of three DCT coefficients near medium frequency. Secondly, the sensitivity of the prediction mode to malicious attacks and recompression operations is different. Therefore, the algorithm classifies the prediction mode of the macroblock and generates the authentication code using the prediction mode. The experimental results show that the video quality is negligible affected after watermark embedding, and the change of video bit rate is basically constant. At the same time, the chromatic DCT algorithm can also use the authentication code of the prediction mode to detect and locate the tampering at the 4 x 4 sub block level with good robustness.