The position and orientation of an object embedding with a permanent magnet can be acquired in real time via a magnetic tracking system. However, one characteristic of the magnetic tracking technique is its varied tracking accuracy along with the tracking distance. Hence, this paper tends to investigate the relationship between the tracking accuracy and the distance from the magnet to the sensor array by both simulations and experiments. Results show that the relationship is expressed as a cubic polynomial equation, and that the equation coefficients are related to the properties of the magnet and the signal-to-noise ratio of the sensor outputs. When the magnet is located at a distance from 36 to 96 mm above the sensor array, the system had the best performance, while average localization and orientation errors were 0.70 mm and 1.22°, respectively. Thus, this system achieved the best tracking accuracy compared with the state-of-the-art of magnetic tracking systems. This paper is helpful to the researchers who want to implement a magnetic tracking system or need to know clearly the valid tracking distance of a magnetic tracking system.