High degrees of interaction fidelity (IF) in virtual reality (VR) are said to improve user experience and immersion, but there is also evidence of low IF providing comparable experiences. VR games are now increasingly prevalent, yet we still do not fully understand the trade-off between realism and abstraction in this context. We conducted a lab study comparing high and low IF for object manipulation tasks in a VR game. In a second study, we investigated players' experiences of IF for whole-body movements in a VR game that allowed players to crawl underneath virtual boulders and "dangle'' along monkey bars. Our findings show that high IF is preferred for object manipulation, but for whole-body movements, moderate IF can suffice, as there is a trade-off with usability and social factors. We provide guidelines for the development of VR games based on our results.