The visual processing of behaviorally relevant stimuli is enhanced through top-down attentional feedback. One possibility is that feedback targets early visual areas first and the attentional enhancement builds up at progressively later stages of the visual hierarchy. An alternative possibility is that the feedback targets the higher-order areas first and the attentional effects are communicated “backward” to early visual areas. Here, we compared the magnitude and latency of attentional enhancement of firing rates in V1, V2, and V4 in the same animals performing the same task. We found a reverse order of attentional effects, such that attentional enhancement was larger and earlier in V4 and smaller and later in V1, with intermediate results in V2. These results suggest that attentional mechanisms operate via feedback from higher-order areas to lower-order ones.