Both the existing soil fertility and fertilizer application have important influences on crop yield and nitrogen (N) use efficiency, but knowledge about their interaction is still lacking. The interaction of N application and integrated soil fertility index on grain yield and N fertilizer use efficiency such as N recovery efficiency, soil N dependency rate, and N contribution rate of wheat was studied through 2-year in situ tests in Changshu County of the Taihu Lake region, China. The results indicated that there are desirable quantitative relationships among the assessment indices, N application level and integrated soil fertility index. The regression coefficients showed that the ratio of the relative contribution of soil fertility and N application level to wheat yield was 0.48:1, and their contribution ratios to fertilizer N utilization efficiencies were 1.10:1, 1.45:1 and 1.93:1, respectively. The effects of both soil fertility and N application level on yield and N recovery efficiency indicated that it is difficult to simultaneously improve yield and N recovery efficiency of wheat by improving soil fertility when attempting to maximize yield. Only by enhancing soil fertility, reducing the N application level and decreasing the maximum yield, can high levels of N recovery efficiency be achieved. For this study, a ratio of N application level decrement to integrated soil fertility increment greater than 0.89 was necessary to increase the N recovery efficiency to 55%. Under this optimal condition, the reduction in N application level was 16%, and the mean yield loss of wheat was only 8%.