SiO2 and phosphosilicate glass (PSG) films containing Si nanocrystals (nc-Si) as small as a few nanometers were studied by electron spin resonance (ESR) and photoluminescence (PL), and the correlation between the two measurements was examined. It is shown that the incorporation of nc-Si in SiO2 results in the drastic increase in the ESR signal; the signal is assigned to the Si dangling bonds at the interfaces between nc-Si and matrices (P-b centers). The ESR signal becomes weaker by doping P into SiO2 matrices, i.e., by using PSG as matrices. By increasing the P concentration, the ESR signal decreases further. By decreasing the ESR signal, the low-energy PL peak at 0.9 eV decreases, while the band-edge PL at 1.4 eV increases. These results suggest that the 0.9 eV peak is related to P-b centers, and that the decrease in the density of the P-b centers by P doping brings about an improvement in the band-edge PL efficiency of nc-Si. (C) 2000 American Institute of Physics. [S0021-8979(00)07104-8].