The lack of low‐loss and high‐efficiency nonreciprocal isolators has become one of the limitations in the development of terahertz (THz) application systems. This work demonstrates that the longitudinally magnetized InSb can achieve one‐way transmission for one photonic spin state but not for linear polarization (LP) state due to the chiral mirror‐symmetry of the two spin states. To solve this issue, a silicon microstructure is fabricated on the InSb substrate to form a magnetoplasmon/dielectric metasurface, where both the time‐reversal and mirror‐reversal symmetric transmission of the two spin states can be broken. In this device, the forward LP state is efficiently transformed into one of the spin states and output with low loss, but the backward wave is forbidden, which achieves the one‐way transmission for the LP incidence with over 30 dB isolation and only 1.7 dB insertion loss. When a THz polarizer is added behind the device to obtain the LP output, the isolation can reach 40 dB, significantly better than the previous reports. This study is significant to understand the nonreciprocal transmission and manipulation mechanism of THz spin states in the magnetized semiconductor and promotes the development of high‐performance THz isolators under the weak magnetic field.