Exposure to early adversity (EA) is associated with long-lasting dysregulations in cognitive processes sustained by brain regions that are sensitive to stress hormones: the hippocampus, the amygdala, and the prefrontal cortex. The Life Cycle Model of Stress highlights the importance of considering the timing at which EA began, as these brain regions follow distinct developmental trajectories. We aimed to test this hypothesis by assessing whether adults exposed to EA exhibit different cognitive patterns as a function of the age at which they were first exposed to EA. Eighty-five healthy men and women aged 21–40 years old (y/o) exposed to EA, as assessed by the Adverse Childhood Experience Questionnaire, were grouped based on the age of first exposure to EA: 0–2 y/o (“Infancy”: hippocampal development), 3–7 y/o (“Early childhood”: amygdala development) and after the age of 8 (“Childhood/Adolescence”: frontoamygdala connectivity development). Declarative memory, attentional bias to threat and emotion regulation were measured. Results revealed increased attentional bias to threat in women first exposed to EA after 8 years. This result is in line with the Life Cycle Model of Stress and highlights the importance of considering the age at exposure to EA when investigating the effects of EA on cognitive processes.