To evaluate the contribution that dry matter and nitrogen (N) accumulation make when translocated to grain during the grain-filling period between genotypes with high and low N harvest index (NHI), a pot experiment was conducted in the greenhouse under a uniform environment. Three high-NHI rice varieties (Yangdao6, R26, and R138) and three low-NHI varieties (Ceysvonic, R44, and R98) were used in our experiment. Significant differences in dry matter and N accumulation, partitioning, and translocation were found between high- and low-NHI genotypes. Compared with low-NHI genotypes, high-NHI genotypes had greater amount of dry matter and N translocation to grain, and higher dry matter and N translocation efficiency, on the other hand, high-NHI genotypes had the lower contribution ratios of pre-anthesis assimilate and N to the grain. Regression analysis showed that grain yield was closely associated with dry matter and N accumulation at heading and maturity stage, N and dry matter translocation in high-NHI genotypes. On the other hand, grain yield was closely associated with the contribution of pre-anthesis assimilate and N to grain in low-NHI genotypes. These suggested that pre- and post-anthesis assimilate and N accumulations, including assimilate and N translocations, were very important in improving the grain yield in high-NHI genotypes.