Humans have an impressive ability to discriminate between faces despite their similarity as visual patterns [1, 2]. This expertise relies on configural coding of spatial relations between face features and/or holistic coding of overall facial structure [2–6]. These expert face-coding mechanisms appear to be engaged most effectively by upright faces, with inverted faces engaging primarily feature-coding mechanisms [2, 7–11]. We show that opposite figural aftereffects can be induced simultaneously for upright and inverted faces, demonstrating that distinct neural populations code upright and inverted faces. This result also suggests that expert (upright) face-coding mechanisms can be selectively adapted. These aftereffects occur for judgments of face normality and face gender and are robust to changes in face size, ruling out adaptation of low-level, retinotopically organized coding mechanisms. Our results suggest a resolution of a paradox in the face recognition literature. Neuroimaging studies have found surprisingly little orientation selectivity in the fusiform face area (FFA) despite evidence that this region plays a role in expert face coding [12–14] and that expert face-coding mechanisms are selectively engaged by upright faces [2, 7–11]. Our results, demonstrating orientation-contingent adaptation of face-coding mechanisms, suggest that the FFA's apparent lack of orientation selectivity may be an artifact of averaging across distinct populations within the FFA that respond to upright and inverted faces.