Transition metal-based phosphides have been increasingly developed as catalysts for efficient water splitting. However, their catalytic performance for oxygen evolution still needs further improvement through incorporating new physical and chemical effects. Herein, for the first time, we constructed phosphorus vacancies in cobalt phosphide integrated with reduced graphene oxide (R-CoPx/rGO) as a precursor template for defect engineering for efficient water oxidation. This template in situ transforms into the defect- and lattice distortion-enriched β-CoOOH integrated with rGO (R-CoPx/rGO(O)), which simultaneously results in significantly enhanced oxygen evolution activity and stability. This catalyst with a 3D hierarchical nanostructure and defect-enriched surface exhibits a small overpotential of 268 mV (10 mA cm−2) and high current density of 311 mA cm−2 at 1.8 V vs. RHE for oxygen evolution, outperforming the precious metal RuO2 catalyst. Density functional theory (DFT) calculations reveal that the oxygen defects induced by phosphorus vacancies can promote the electrical conductivity of β-CoOOH. Furthermore, we prove the essential role of the unique β-CoOOH for efficient water oxidation via a carbon layer coating strategy. This study provides a new path for the development of metal phosphides as precursors for high performance OER catalysts.