This study examines the combined effects of sleep duration and exercise habits on all-cause mortality in older Chinese adults using data from the Chinese Longitudinal Healthy Longevity Survey (CLHLS). Methods Data were collected from 7231 residents aged 60 and above from the CLHLS. Participants were categorized based on their sleep duration (short sleep: <6 hours, normal sleep: 6-8 hours, and long sleep: >8 hours) and exercise habits (physically active, physically inactive, inactive-to-active, and active-to-inactive). The analysis was conducted over three follow-up periods (2011, 2014, and 2018). Cox proportional hazards regression models were used to assess the associations between sleep duration, exercise habits, and all-cause mortality. Results The results showed that compared to short sleepers, individuals with normal sleep (6-8 hours) had a slightly reduced risk of death, although this reduction was not statistically significant (HR = 0.97, 95% CI 0.87-1.08). In contrast, long sleepers (>8 hours) had a significantly higher risk of mortality (HR = 1.28, 95% CI 1.16-1.43). Regarding exercise habits, regular exercise or transitioning from inactivity to exercise significantly reduced mortality risk compared to those who did not exercise (HR = 0.74, 95% CI 0.66-0.83 and HR = 0.88, 95% CI 0.80-0.97, respectively). Notably, exercise habits did not significantly impact mortality among short sleepers, regardless of gender. However, both men and women with normal or long sleep experienced significant mortality benefits from regular exercise. Additionally, older women who transitioned from a sedentary lifestyle to physical activity during periods of long sleep demonstrated lower mortality rates. Conclusion Both sleep duration and exercise habits are associated with mortality risk among older Chinese adults, with notable gender differences in their combined effects. Targeted health policies that encourage improved sleep and exercise habits, while considering gender-specific needs, are essential to reduce mortality and enhance quality of life in this population.