Urbanization is a trend lasting for more than one century worldwide. Four hundred ninety male and female adult Chinese Han participants with different urban and rural childhoods were included in this study. Early-life urban environment was found benefit for total grey matter volume (GMV), dorsolateral prefrontal cortex (DLPFC) GMV, temporal pole (TP) GMV and cognition function, and negatively correlated with medial prefrontal cortex (MPFC) GMV. Regression analysis showed that maternal education was a protective factor for total and DLPFC GMVs, while having siblings was better for MPFC GMV. Total, DLPFC and TP GMVs acts mediation effects between childhood urbanicity and different cognitive domains. These findings may suggest some pros and cons on brain structure associated with childhood urbanicity and related environmental factors.