To identify possible causes for the long‐term trends in nutrient concentrations in the southern Japan Sea (JS), we studied nutrient concentrations that were obtained by the Japan Meteorological Agency. Our evaluation shows that phosphate concentrations declined in the surface layers in summer (0–20 and 21–50 m depth) and winter (0–20, 21–50, and 51–100 m depth) over the last 40 years, while no significant linear trend was observed for nitrate concentrations. The declining trend in the phosphate concentration was quantified as 1.8–3.3 nM yr⁻¹. The increase in atmospheric nutrient deposition to the JS could not explain the decline in phosphate concentration. In addition, the mixed‐layer depth during winter did not demonstrate any significant trend, and an increase in phosphate concentrations was not observed in any layers; therefore, the decrease in nutrient supply from deep JS water was not considered a major possible cause for the decline in the phosphate concentration. In contrast, the phosphate concentration in the surface of the southern JS during winter showed a significant positive correlation with the concentration in the 21–50 m depth layer of the saline East China Sea (ECS) water in the preceding summer, and the surface water of the southern JS was almost entirely replaced by water originating from the ECS during May–October. Therefore, it is concluded that the declining trend in the phosphate concentrations in the southern JS is caused by horizontal advection of ECS water.