Background: The apolipoprotein E (APOE) gene is the most potent genetic risk factor for dementia. However, there are few studies on how the APOE gene affects cognitive domain functions. Objective: This study aimed to investigate the effects of risk factors for dementia on cognitive function in patients with mild cognitive impairment and Alzheimer's disease (AD). Methods: This study included subjects whose Clinical Dementia Rating scores ranged from 0.5 to 2 and who were older than 65 years. Risk factors for dementia included the APOE epsilon 4 allele, age, education period, employment period, body mass index, and exercise. APOE genotyping was performed by polymerase chain reaction, and other factors were identified using medical charts or structured checklists. Cognitive function was measured using the Seoul Neuropsychological Screening Battery II. Results: General cognitive function did not show a significant difference according to APOE epsilon 4 status. However, the score for delayed verbal memory was lower in the APOE epsilon 4-carrier group than in the non-carrier group (p < 0.05). In addition, age, education period, employment period, and exercise were correlated with different cognitive function domains in the non-carrier group (p < 0.05); however, the carrier group was showed a significant correlation between age, body mass index, and cognitive domains. Conclusion: Our findings suggest that APOE epsilon 4 significantly decreases verbal memory in patients with AD. Moreover, the effects of risk factors on cognitive function were significantly different according to the APOE epsilon 4 status.