Being a victim of bullying or cyberbullying can be linked to participating less in physical activity. Considering that physical self-concept can play an important role in the relationship between both constructs, this study aims to analyse the mediating effect of physical self-concept between victimization/cybervictimization and physical activity, considering the moderating effect of gender. A total of 870 schoolchildren aged 12 to 19 from southern Spain participated in this explanatory cross-sectional study. We studied the variables of victimization, cyber-victimization, physical self-concept and organized physical activity through self-reports, and analysed the moderated mediation using the Process macro. Negative relationship was found between physical activity and both victimization and cybervictimization, and this association was mediated by physical self-concept. In turn, moderating analyses showed that physical self-concept had a stronger relationship on physical activity in boys. In conclusion, a positive physical self-concept can mediate in victims of bullying or cyberbullying opting not to participate in physical activities, especially in boys.