To evaluate how age and apolipoprotein E-ε4 (APOE4) status interact with APOE-independent polygenic risk score (PRSnon-APOE), we estimated PRSnon-APOE in superagers (age ≥ 90 years, N = 346), 89- controls (age 60-89, N = 2930), and Alzheimer’s disease (AD) cases (N = 1760). Using superagers, we see a nearly 5 times greater odds ratio (OR) for AD comparing the top PRSnon-APOE decile to the lowest decile (OR = 4.82, p = 2.5 × 10-6), which is twice the OR as using 89- controls (OR = 2.38, p = 4.6 × 10-9). Thus PRSnon-APOE is correlated with age, which in turn is associated with APOE. Further exploring these relationships, we find that PRSnon-APOE modifies age at onset among APOE4 carriers, but not among noncarriers. More specifically, PRSnon-APOE in the top decile predicts an age at onset 5 years earlier compared with the lowest decile (70.1 vs. 75.0 years; t-test p = 2.4 × 10-5) among APOE4 carriers. This disproportionally large PRSnon-APOE among younger APOE4-positive cases is reflected in a significant statistical interaction between APOE4 status and age at onset (β = -0.02, p = 4.8 × 10-3) as a predictor of PRSnon-APOE. Thus, the known AD risk variants are particularly detrimental in young APOE4 carriers.