Background As the population ages, the health of rural older adults is of increasing concern to society. Pain, decreased activity of daily living, depression, and sleep are important factors affecting the quality of life of older adults. This study aimed to explore the complex relationship between pain, activity of daily living, depression, and sleep in rural older adults, with the goal of providing new perspectives and intervention strategies to improve sleep quality. Methods This study was based on the data from the 2020 China Health and Retirement Longitudinal Study, and rural older adults aged 60 years and above were selected as the study subjects, with a final sample size of 5,352. Stata 18.0 and SPSS 27.0 software were used for statistical analysis, and t-tests, analysis of variance (ANOVA), and Pearson correlation analyses were used for one-way analyses, and PROCESS 4.2 was used for mediation effect analysis and testing. Results Pain in older adults was negatively correlated with sleep duration (r = -0.212) and positively correlated (p < 0.001) with impairment to activity of daily living (r = 0.339) and depression (r = 0.355). The mediation test reported that pain in older adults had a direct effect on sleep duration (95% CI: -0.076 to -0.043), with activity of daily living (95% CI: -0.014 to -0.004) and depression (95% CI: -0.026 to -0.017) acting as chained mediators between the two. Conclusion This study reveals the interrelationships between pain, activity of daily living, depression and sleep in rural older adults. It is recommended that medical resources be strengthened, health awareness be increased, community care services be improved, recreational activities be provided, and family emotional support be encouraged to improve the health and quality of life of older adults.