The objective of this study was to evaluate binder characterization of the desulfurized rubber asphalt (DRA) and crumb rubber asphalt (CRA) with different contents and compare those materials with available virgin asphalt (VA). The effects of desulfurized rubber and regular rubber on VA were investigated from the physical, rheological and stability properties point of view. The results indicated that DRA had the similar physical and rheological performances with CRA, but DRA was much better in improving storage property. The desulfurized rubber could integrate into asphalt to perform both chemical reaction and physical swelling while crumb rubber mainly dispersed incompatibly in asphalt. The chemical fusion in DRA could provide much better rutting resistance and stability performance in high temperature and increase the low-temperature performance. Although the asphalt characterization was enhanced with increasing rubber contents, excessive rubber addition could not get much better performance improvement. It was recommended that 15–20% desulfurized rubber content was optimum for designing asphalt rubber mixture to acquire improved performance and long storage stability. Overall, the long storage of DRA would provide asphalt rubber technology a much wider range of service and adaption in the road field construction.