Understanding socioeconomic disparities in physical activity is important, given its contribution to overall population-wide health and to health disparities. Existing studies examining trends in these disparities have focused exclusively on physical activity during leisure-time and have not investigated the potential moderators of socioeconomic disparities in physical activity. Using self-reported data from the US National Health and Nutrition Examination Survey (NHANES) 2007 to 2016 for 29,039 adults aged 20 years and over we examined education-related disparities in overall (total) moderate-to-vigorous intensity physical activity, and in its sub-components, recreational (leisure-time) and non-recreational (active transportation and work) activity. We also examined if education-related disparities in physical activity were moderated by age, gender, and race/ethnicity. Logistic regression models were used to evaluate disparities in physical activity according to education group and their moderation across age, gender, race/ethnicity, and time-period. Overall activity levels (% ≥150 min/week) were highest amongst highly educated adults, yet contrasting education-related disparities were found for recreational and non-recreational activities (active transportation and work), favoring the highest- and lowest-educated groups respectively. Within each domain of activity, associations were moderated by age and race/ethnicity, and by gender for work-based activity. The net result was that education-related disparities in total activity were substantially larger in older adults (P < 0.001) and amongst women (P < 0.001). For example, the estimated difference in the probability of being active in the highest versus the lowest educational groups was 23.1% (95% CI: 19.1, 27.2) amongst those aged ≥60 years, yet 10.8% (95% CI: 7.1, 14.6) amongst those aged 20–39. Education-related disparities in physical activity persisted from 2007 to 2016. Our results suggest that understanding and addressing these disparities requires assessment of their multiple domains, and identification of the demographic sub-groups for which the disparities are more or less pronounced.