Application of carbon fiber-reinforced polymer (CFRP) composites has been promoted successfully for strengthening or repairing concrete structural members. However, if a column member is strengthened with longitudinal FRP sheets in which fibers are laid parallel to the column axis, the FRP composite will soon buckle under compression to ultimately debond off the concrete surface. It is the objective of the current research to investigate the effect of the recently developed grooving method (GM) on cylindrical specimens strengthened with FRP sheets under compressive loading, and to determine whether GM postpones the buckling of FRP sheets and their debonding off the concrete substrate. For the purposes of this study, 35 cylindrical reinforced concrete specimens, 150 mm in diameter and 500 mm in height, were tested under compressive loading with eccentricities of zero, 30, 60, and 90 mm. The specimens were additionally subjected to the four-point flexural bending test to represent infinity eccentricity. Fiber-reinforced polymer (FRP) sheets with longitudinal fibers parallel to the column axis were installed using the conventional externally bonded reinforcement (EBR) and grooving method, taking advantage of the special technique of externally bonded reinforcement on grooves (EBROG). Experimental results revealed that the grooving method was much more effective than the EBR in increasing the load-carrying capacity of the specimens. Moreover, it was found that longitudinal fibers exhibited an enhanced effect with increasing load eccentricity. The results showed that in the columns strengthened with longitudinal FRP strips using the EBROG method, the load-carrying capacity enhancements of 8.3, 12.2, 25.8, 36.0, and 53.3% relative to those of the controls were achieved for 0, 30, 60, 90 mm, and infinity eccentricities, respectively.