As in the lightless conditions, the ordinary intelligent video surveillance system can not detect the target in the darkness, but the infrared-based intelligent video surveillance system could overcome the shortcoming and monitor the sites for 24 hours. This paper introduces the technology of intelligent video analysis based on infrared, which includes infrared image preprocessing, moving target detection and tracking. The components and functions of the intelligent video surveillance system are also introduced in this paper.