Lucerne (Medicago sativa L.) is one of the most successfully introduced species for revegetation on the Loess Plateau of China and provides important ecosystem services. However, the driving mechanism of soil organic carbon (SOC) and total nitrogen (TN) in lucerne grasslands remains unclear. This study explored the controlling factors of SOC and TN in lucerne grasslands in the semiarid Loess Plateau. A total of 112 quadrats were employed in 28 lucerne fields. Vegetation characteristics, topographic factors, and soil properties at a 0–20 cm depth were measured in each quadrat. The SOC and TN contents increased with altitude and showed positive correlations with species richness, aboveground biomass of native plants, soil moisture, soil inorganic nitrogen, total soil phosphorus (P), and C:P and N:P ratios. Variations in SOC and TN contents were mainly attributed to soil resources, followed by the interaction of topography, vegetation and soil. Soil P, soil moisture, altitude, and native plant species were the main factors controlling SOC and TN contents in these lucerne grasslands. Results suggest that specific abiotic (soil P and moisture) and biotic (plant species diversity) factors controlled SOC and TN in semiarid lucerne grasslands. These factors should be included in SOC and TN evaluation models to predict the future terrestrial ecosystem carbon and nitrogen dynamics.