High porosity with interconnected voids between aggregate particles is the primary characteristic of portland cement pervious concrete (PCPC), which, however, causes a significant decrease in its strength and abrasion resistance. In this study, latex and fiber were added to improve the abrasion resistance of PCPC mixtures. Laboratory tests were conducted to evaluate the performance of latex-modified pervious concrete with a particular focus on abrasion resistance. Test results show that adding latex desirably improved strength and abrasion resistance of PCPC, whereas fiber did not show a significant effect on the mechanical properties of PCPC. In addition, the asphalt pavement analyzer (APA) abrasion test was found to be feasible for evaluating the abrasion resistance of pervious concrete.