Machine learning algorithms can be trained to estimate age from brain structural MRI. The difference between an individual’s predicted and chronological age, predicted age difference (PAD), is a phenotype of relevance to aging and brain disease. Here, we present a new deep learning approach to predict brain age from a T1-weighted MRI. The method was trained on a dataset of healthy Icelanders and tested on two datasets, IXI and UK Biobank, utilizing transfer learning to improve accuracy on new sites. A genome-wide association study (GWAS) of PAD in the UK Biobank data (discovery set: , replication set: ) yielded two sequence variants, rs1452628-T (, ) and rs2435204-G (, ). The former is near KCNK2 and correlates with reduced sulcal width, whereas the latter correlates with reduced white matter surface area and tags a well-known inversion at 17q21.31 (H2).