The human visual system converts identically sized retinal stimuli into different-sized perceptions. For instance, the Müller-Lyer illusion alters the perceived length of a line via arrows attached to its end. The strength of this illusion can be expressed as the difference between physical and perceived line length. Accordingly, illusion strength reflects how strong a representation is transformed along its way from a retinal image up to a conscious percept. In this study, we investigated changes of effective connectivity between brain areas supporting these transformation processes to further elucidate the neural underpinnings of optical illusions. The strength of the Müller-Lyer illusion was parametrically modulated while participants performed either a spatial or a luminance task. Lateral occipital cortex and right superior parietal cortex were found to be associated with illusion strength. Dynamic causal modeling was employed to investigate putative interactions between ventral and dorsal visual streams. Bayesian model selection indicated that a model that involved bidirectional connections between dorsal and ventral stream areas most accurately accounted for the underlying network dynamics. Connections within this network were partially modulated by illusion strength. The data further suggest that the two areas subserve differential roles: Whereas lateral occipital cortex seems to be directly related to size transformation processes, activation in right superior parietal cortex may reflect subsequent levels of processing, including task-related supervisory functions. Furthermore, the data demonstrate that the observer's top–down settings modulate the interactions between lateral occipital and superior parietal regions and thereby influence the effect of illusion strength.