The ApoE ε4 allele is the most significant genetic risk factor for late-onset Alzheimer disease. The risk conferred by ε4, however, differs across populations, with populations of African ancestry showing lower ε4 risk compared to those of European or Asian ancestry. The cause of this heterogeneity in risk effect is currently unknown; it may be due to environmental or cultural factors correlated with ancestry, or it may be due to genetic variation local to the ApoE region that differs among populations. Exploring these hypotheses may lead to novel, population-specific therapeutics and risk predictions. To test these hypotheses, we analyzed ApoE genotypes and genome-wide array data in individuals from African American and Puerto Rican populations. A total of 1,766 African American and 220 Puerto Rican individuals with late-onset Alzheimer disease, and 3,730 African American and 169 Puerto Rican cognitively healthy individuals (> 65 years) participated in the study. We first assessed average ancestry across the genome (“global” ancestry) and then tested it for interaction with ApoE genotypes. Next, we assessed the ancestral background of ApoE alleles (“local” ancestry) and tested if ancestry local to ApoE influenced Alzheimer disease risk while controlling for global ancestry. Measures of global ancestry showed no interaction with ApoE risk (Puerto Rican: p-value = 0.49; African American: p-value = 0.65). Conversely, ancestry local to the ApoE region showed an interaction with the ApoE ε4 allele in both populations (Puerto Rican: p-value = 0.019; African American: p-value = 0.005). ApoE ε4 alleles on an African background conferred a lower risk than those with a European ancestral background, regardless of population (Puerto Rican: OR = 1.26 on African background, OR = 4.49 on European; African American: OR = 2.34 on African background, OR = 3.05 on European background). Factors contributing to the lower risk effect in the ApoE gene ε4 allele are likely due to ancestry-specific genetic factors near ApoE rather than non-genetic ethnic, cultural, and environmental factors.