The study of information-processing adaptations in the brain is controversial, in part because of disputes about the form such adaptations might take. Many psychologists assume that adaptations come in two kinds, specialized and general-purpose. Specialized mechanisms are typically thought of as innate, domain-specific, and isolated from other brain systems, whereas generalized mechanisms are developmentally plastic, domain-general, and interactive. However, if brain mechanisms evolve through processes of descent with modification, they are likely to be heterogeneous, rather than coming in just two kinds. They are likely to be hierarchically organized, with some design features widely shared across brain systems and others specific to particular processes. Also, they are likely to be largely developmentally plastic and interactive with other brain systems, rather than canalized and isolated. This article presents a hierarchical model of brain specialization, reviewing evidence for the model from evolutionary developmental biology, genetics, brain mapping, and comparative studies. Implications for the search for uniquely human traits are discussed, along with ways in which conventional views of modularity in psychology may need to be revised.