In frequency-modulation (FM)-based passive radar, the strong side peaks randomly appearing in the ambiguity function will generate a false alarm of target detection. To mitigate this side peak interference, this paper starts with a detailed analysis of the structure and ambiguity function of FM stereo signal. Then, it expounds the formation and characteristics of side peaks, together with a side peak identification method, which identifies and discards the false detections of the same range and Doppler caused by side peaks, respectively. The performance analysis, conducted using both simulated data and real recorded datasets, proves that the proposed method can eliminate the false targets caused by side peaks thus improving the detection performance in FM-passive radar.