Guiding farmers to respond to climate change is of great significance to China’s adaptation to climate change and global sustainable development. Using survey data of 540 farmers from Sichuan Province, the binary probit model and propensity score matching (PSM) method were used to analyze the peer effects on farmers’ response to climate change. The results show that (1) the adaptive behaviors of relatives and friends, neighbors, and government have a significant positive effect on farmers’ response to climate change; (2) the adaptability of different groups to climate change has different impacts on farmers’ response to climate change, and the order of impacts is government > relatives and friends > neighbors; (3) farmers with smaller per capita arable land and lower per capita income are more inclined to respond to climate change under the influence of their relatives and friends’, neighbors’, and governments’ adaptation to climate change; and (4) social trust and environmental perception play a significant mediating role in the peer effects of farmers’ response to climate change. This study will provide reference for the government to guide farmers in making policies related to climate change.