Observational studies strongly supported the association of low levels of circulating 25-hydroxyvitamin D (25OHD) and cognitive impairment or dementia in aging populations. However, randomized controlled trials have not shown clear evidence that vitamin D supplementation could improve cognitive outcomes. In fact, some studies reported the association between vitamin D and cognitive impairment based on individuals aged 60 years and over. However, it is still unclear that whether vitamin D levels are causally associated with Alzheimer’s disease (AD) risk in individuals aged 60 years and over. Here, we performed a Mendelian randomization (MR) study to investigate the causal association between vitamin D levels and AD using a large-scale vitamin D genome-wide association study (GWAS) dataset and two large-scale AD GWAS datasets from the IGAP and UK Biobank with individuals aged 60 years and over. Our results showed that genetically increased 25OHD levels were significantly associated with reduced AD risk in individuals aged 60 years and over. Hence, our findings in combination with previous literature indicate that maintaining adequate vitamin D status in older people especially aged 60 years and over, may contribute to slow down cognitive decline and forestall AD. Long-term randomized controlled trials are required to test whether vitamin D supplementation may prevent AD in older people especially those aged 60 years and may be recommended as preventive agents.