This study reconciles the contrasting evidence in the fiscal decentralization-government size nexus by hypothesizing that the effect of fiscal decentralization is conditional on the distribution of government size. Using panel quantile regression with non-additive fixed effects and with instrumental variable, we find evidence to our hypothesis for sixty-five countries across four decades. Fiscal decentralization's impact on government size is negative in the lower quantiles due to inter-jurisdictional competition, while it is positive in the middle and upper quantiles owing to the efficiency hypothesis. The nature of heterogeneity is robust across an alternate and more composite decentralization indicator of fiscal autonomy as well. By accounting for ethnic fragmentation and democracy as transmission channels, this heterogeneous pattern is consolidated. Thus, the findings suggest the importance of methodological refinement and socio-political transmission channels in explaining the equivocal relationship between fiscal decentralization and government size.