The cables of the long-span bridge are usually featured as ultra-low frequency, hence making the acceleration unable to accurately capture the information, e.g. damping ratios, for assessing the cable state assessment and mitigating the excessive structural vibration. The displacement was approved to be more sensitive to the low-frequency vibration than the acceleration. However, there is still a lack of effective method to accurately monitor the long-term displacements of bridge cables using reference-free methods. To address this issue, this paper develops a novel acceleration-based approach for monitoring the long-term displacements of the cables of long-span bridges. In the monitoring scheme, recursive least squares method is utilized to conduct baseline correction in the time domain integration of acceleration. An adaptive band-pass filtering method considering cable vibration characteristics is used to eliminate noise, thus avoiding the difficulty of selecting the cut-off frequency by experience in traditional methods. A numerical test of an analytical cable model and a field experiment of the hanger of a full-scale suspension bridge are applied to the applicability and robustness of the developed method. Result shows that adaptive band-pass filter considering the vibration characteristics is suitable for estimating the displacements of the cables. The estimated displacements using the developed method agree well with the background truth in both time and frequency domains.