We extend the phase field crystal (PFC) framework to quantitative modeling of polycrystalline graphene. PFC modeling is a powerful multiscale method for finding the ground state configurations of large realistic samples that can be further used to study their mechanical, thermal, or electronic properties. By fitting to quantum-mechanical density functional theory (DFT) calculations, we show that the PFC approach is able to predict realistic formation energies and defect structures of grain boundaries. We provide an in-depth comparison of the formation energies between PFC, DFT, and molecular dynamics (MD) calculations. The DFT and MD calculations are initialized using atomic configurations extracted from PFC ground states. Finally, we use the PFC approach to explicitly construct large realistic polycrystalline samples and characterize their properties using MD relaxation to demonstrate their quality.