Transboundary pollution in the watersheds poses serious challenges to the allocation of water resources, social stability, and economic sustainability of upstream and downstream cities. Water pollution in the basin has become a key factor restricting regional economic and social development. To address this problem, a watershed ecological compensation model was proposed. This model includes a range of indexes that reflect upstream and downstream government's decision evolution process in factors such as ecological benefits, the marginal benefits of reputation, opportunity costs and direct costs, and compensation fees. Also, the model incorporates three-way decision theory to expand government's strategies when treating the following: water pollution problems, evolutionary game theory to capture decision dynamics for government with different strategies, the influence of reputation and aim to reduce the regulatory pressure of the central government. In the simulation, the proposed ecological compensation model predicts decisions of the upstream and downstream government with various water problems. Eventually, a case study of HaiHe River Basin in China serves to verify the practicable effectiveness of the foregoing model combined with numerical simulation and support key insights as below: (i) when the reputation of the upstream government increases or decreases, it will form an effective positive feedback mechanism to promote its enthusiasm for protecting water resources; (ii) for the whole ecosystem, setting a reasonable ecological compensation fee can effectively promote active cooperation between upstream and downstream governments; (iii) as long as upstream government choose a non-commitment strategy, downstream government will be less willing to pay compensation fees. This study provides a valid approach that can be applied to develop a long-term effective mechanism for protecting river basin sustainability and improving the system of ecological compensation in the watersheds.