Background This study is aimed to analyze the prognostic factors affecting the short-term efficacy of non-surgical treatment of patients in periodontitis from stage II to stage IV by the multilevel modeling analysis. Materials and methods A total of 58 patients with chronic periodontitis were included in this study. Patients were clinically explored before and 3 months after the treatment and the difference in probing depth was determined [Reduction of probing depth (Delta PD) = baseline PD - finial probing depth (FPD)] which is considered as the therapeutic evaluation. Three different levels were analyzed: patients, teeth and sites to construct a multi-layer linear model. Results Probing depth (PD) improved significantly compared with that before treatment (p < 0.05), in which FPD was (3.90 +/- 1.39) mm, and the Delta PD was (1.79 +/- 0.97) mm. Compared with the mesial sites and distal sites of the multi-rooted teeth, the number of PD >= 5 mm or PD < 5 mm after the treatment was significantly different (P < 0.05), and the proportion of PD < 5 mm was higher in mesial sites. The null model showed that Delta PD varied greatly between groups at various levels (P < 0.001), with prediction variable of site level, tooth level, and patient level accounted for 66%, 18%, and 16% of the overall difference, respectively. The complete model showed that the Delta PD of smokers was significantly lower than that of non-smokers (P < 0.001). The Delta PD of the mesial and distal sites was larger than that of the buccolingual central site (P < 0.001). The Delta PD of single-rooted teeth was larger than that of multi-rooted teeth (P < 0.001). The baseline PD, tooth mobility (TM), bleeding index (BI), clinical attachment loss (CAL) were significantly negatively correlated with Delta PD (P < 0.001). Conclusions Patients with periodontitis from stage II to stage IV, who were non-smoking, have good compliance, good awareness of oral health, and low percentage sites with PD >= 5 mm at baseline, single-rooted teeth with hypomobility, less clinical attachment loss and lower bleeding index and sites of mesial or distal can obtain an ideal short-term efficacy of non-surgical treatment.