Arbuscular mycorrhizal fungi (AMF), which can form symbiotic associations with many terrestrial plants, are critical for crop yields and agroecosystem sustainability. In this study, we assessed the influence of rice-upland crop rotations on soil AMF diversity and composition. We also explored the mechanisms of rice (Oryza sativa L.)-upland crop rotations that affect AMF using trait-based guild methods. We found that rotations of rice with different plants differentially influenced soil AMF. Rice-wheat (Triticum aestivum L.) (RW) and rice-Chinese milk vetch (Astragalus sinicus L.) (RV) rotations significantly altered the soil AMF composition, with RW and RV significantly increasing and decreasing AMF diversity, respectively, compared with the rice-fallow treatment. In addition, RW and RV affected AMF abundance in intra- and extra-radical portions in different ways. For example, both the RW and RV treatments increased AMF spore density, but decreased AMF colonization rate. Different AMF guilds showed different responses to rice-upland crop rotations. The RW treatment increased the rhizophilic guild by 4.9% and decreased the edaphophilic guild by 27.9%, whereas the RV treatment produced opposite trends. Thus, rice-upland crop rotations changed soil AMF diversity, AMF composition, and trait-based guilds in different ways, and rice yield was mainly correlated with AMF colonization rate.