Associations between polymer morphology and the properties of polymer-modified asphalt (PMA) have been widely reported. Fluorescence microscopy (FM) plays a key role in analyzing the morphology of polymers in PMA. However, two-dimensional (2D) fluorescence images may not reveal the true nature of polymer phase in PMA, because the image is a projection of the three-dimensional (3D) space on a 2D plane. This study reconstructs the 3D polymer phase in PMA by using confocal fluorescence microscopy and image processing techniques. The method is successfully used in analyzing the morphology of polymers in styrene–butadiene–styrene (SBS) PMA. A quantitative comparison is made between the results of the 3D volume fraction and 2D area fraction of the polymer phase. The findings indicate that 3D reconstruction is necessary to understand the true state of the polymer phase in PMA. This research contributes to the body of knowledge in civil engineering materials by developing the method for 3D reconstruction of the polymer phase in PMA and discovering the more realistic state of the polymer phase.