To effectively identify the chatter in milling which had great influence on the quality of workpiece, the chatter recognition model based on Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) and Hidden Markov Model (HHM) was established. Frequency spectrum of cutting force signals which were collected through fixed cycle was estimated by using the method of ESPRIT. The cutting state in the process was determined by spectrum characteristic, and cutting force signals were labeled after the filtering of spindle rotation cycle. HHM parameter was trained by cutting force signals which had class labels, and the parameters of this chatter recognition model were obtained. To realize chatter recognition and monitoring of cutting state, PC based real-time acquisition and processing of cutting force signals were developed and integrated into Open Modular Architecture Controller (OMAC) of machine tool. The result of machining experiment showed that CNC system which integrated the function of measuring force could accurately determine the cutting state in the process of machining based on established model. © 2016, Editorial Department of CIMS. All right reserved.