Corner-supported steel modular buildings were constructed using volumetric modules with the inter-module connections at their corners and similar connections to foundations. The rotary inter-module connections were used to assemble a two-storey building frame in this study for investigation of its the mechanical behavior. Such a full-scale building frame was examined under cyclic loading until structural failure, where the failure mode, hysteretic performance, ductility, stiffness degradation as well as energy dissipation were experimentally clarified. The results indicated that the structure experienced ductile failure with buckling at the end of the ceiling and floor beams as well as the bottom end of lower columns. The rotary inter-module connections were able to transfer the internal forces without failure although gaps were formed between the foundation corner fitting and lower corner fitting. A refined finite element (FE) modelling approach was established and validated through comparisons to experimental results, then the load-displacement curves of such steel modular structure were obtained and analyzed under different axial compression ratios as well as stiffness effect of module-to-foundation connections. Theoretical formula was further derived to estimate the initial lateral stiffness of the upper and lower frame considering semi-rigid inter-module and module-to-foundation connections, showing well agreement with the experimental and numerical results. In this way the load transferring mechanism of such modular structure with the rotary inter-module and module-to-foundation connections can be clarified.