Water pollution caused by improper treatment of acid mine drainage (AMD) is one of the most common problems encountered in the mining process. However, the traditional physical and chemical methods are expensive, and the additional products are difficult to treat. Therefore, seeking a simple and efficient treatment method has become the main challenge in solving AMD. In this study, the efficiency of sulfate removal using sulfate-reducing bacteria (SRB) based on the upflow anaerobic sludge blanket (UASB) reactor and the changes in the sludge microbial community structure before and after the reaction were studied. The properties of SRB, particularly their reducing effect, were studied. Results show that SRB had good reduction performance in the environment with pH 6.5 to 7.5. The best-reducing effect was achieved at 15% inoculum. Under different initial sulfate concentrations, the sulfate removal rate in the UASB reactor reached 80-90%. After the reaction, the species' community abundance was increased, and the species diversity was enhanced. These investigations are conducive to providing new ideas and methods for treating AMD and solving the high sulfate concentrations in AMD.